• Home
  • Technology
  • Gaming
  • Entertainment
  • World & Business
  • Science
  • Sports
  • AI
HomeTechnologyGamingEntertainmentWorld & BusinessScienceSportsAI
AI
Reaction

The case for basic AI research without frontier labs or 10,000-plus GPUs

A poster rejects the idea that important work requires a frontier lab and 10,000-plus GPUs. He says some people he talks to think they should leave AI research because they expect end-to-end research to be automated.

rohan anilRA
Alex DimakisAD
Phillip IsolaPI
26 Sources, 12d ago, first seen 12d ago

TLDR

The poster calls it false to assume important AI research can happen only in frontier or “neo” labs with 10,000-plus GPUs. He says some people he talks to think they should leave the field because they expect end-to-end research to be automated. He argues that today’s tools, open problems and ability to tackle them make this by far the best time in recent history to do AI research.

Combined views

371.9K

26 Sources, first seen 12d ago

4.2K likes161 comments790 saves414 reposts

Combined views

371.9K

26 Sources, first seen 12d ago

4.2K likes161 comments790 saves414 reposts

Sentiment

Positive60.5%39.5%Negative

Based on 113 sentiment-bearing replies from 95 accounts across 9 conversations.

Featured Source

Sentiment

Positive60.5%39.5%Negative

Based on 113 sentiment-bearing replies from 95 accounts across 9 conversations.

Related

A rough calculation puts frontier AI models’ potential annual revenue below $250 billion

A user assumes 100 million customers pay an average of $200 a month, including API spending, to estimate a market below $250 billion a year across all providers.

Linear mode connectivity in neural network training

A user calls linear mode connectivity one of the most underrated features of training neural networks.

Tweeting as prompting practice

A user calls tweeting “really good practice” for prompting.

26 Sources

Dimitris Papailiopoulos@DimitrisPapailOne of the worst forms of brainrot AI has cultivated is pessimism about basic research, ie the idea that important work can only happen inside a frontier/neo lab and only with 10k+ GPUs, so the rest should not even bother. What a bleak way to think about science. And it's false.12d
John Thickstun@jwthickstun@DimitrisPapail I agree with this, but I also find the table stakes cost of basic research in AI has increased a lot. Even 1 modern GPU is expensive. Looking forward, I hope the funding model for academic AI research can start looking more like biology and lab sciences rather than mathematics.12d
Julius Adebayo@juliusadmlwell said.12d
Amin Karbasi@aminkarbasi100%. This is the premise behind Foundation AI @fdtn_ai: we can do frontier research without training the next frontier model from scratch. There are deep, fundamental questions in how to extend these models, understand where they fail, and make them more useful. You don’t need 10,000 GPUs to do important science.12d
Sreeram Kannan@sreeramkannanCo-signed. The era of high-frequency research. The following are common myths: 1. Humans are not needed to do research 2. Cost is so high that only frontier labs can do research. 3. Somehow ai based research is bad for research communities The most useful way to think of ai is **accelerating the timescale of research** It is similar to the transition from low-frequency research to high-frequency research akin to what happened in trading. What took years now takes days. Once it’s known that Euler has blowup, in the next week it was shown that Navier Stokes has blowup. The fluid science is nowhere near done, now, we need to figure out how to get to next steps: forcing removal for example. Compressing years to days and hours should be the goal. The @__alpoge__ and @DimitrisPapail of the world, who do fundamental work but are agent native, terminally online and collaborative will do very well. We need to build tools and technologies to help more scientists make this transition soon. We also need postAGI scientific institutions that operate at this high frequency: systems of collaboration, credit and funding need to adapt to this speed of light. It’s an extraordinarily exciting time. We should collaborate to solve humanity’s hardest problems at the speed of light.12d
Alex Dimakis@AlexGDimakisDimitris has been proving you can do important AI basic research with a few GPUs and some agent max subscriptions. Some examples: ECHO paper, looped transformers, open mementos for context management come in mind.12d
rohan anil@_arohan_What BigToken doesn’t want you to know is that they also read arxiv and X to get new ideas. Models are good at magnitude, direction is still hard. There is some uncertainty around for however long though, but its not today Please continue the amazing works like looping transformers and echo.12d
Anastasios Nikolas Angelopoulos@ml_angelopoulosYep. This is a toxic (and crazy) outlook. There are plenty of important problems that basic research will solve outside of labs. Much of this pessimism btw is created by people that have never experienced excellence in pure research and thus don't have a frame of reference.12d
Deborah Jacob@isthatdebbiejIdk who needs to hear this but being kind, chalant, and relentlessly curious is the real moat12d
Phillip Isola@phillip_isola@DimitrisPapail I agree. Further, the basic research within frontier labs is mostly siloed there. If your goal is to further the world's understanding of intelligence, then I think basic research outside of frontier labs is still the best place to do that.12d
    • Home
    • Technology
    • Gaming
    • Entertainment
    • World & Business
    • Science
    • Sports
    • AI
    Dimitris PapailiopoulosChenhao Tan

    26 Sources

    Dimitris Papailiopoulos@DimitrisPapailOne of the worst forms of brainrot AI has cultivated is pessimism about basic research, ie the idea that important work can only happen inside a frontier/neo lab and only with 10k+ GPUs, so the rest should not even bother. What a bleak way to think about science. And it's false.12d
    John Thickstun@jwthickstun@DimitrisPapail I agree with this, but I also find the table stakes cost of basic research in AI has increased a lot. Even 1 modern GPU is expensive. Looking forward, I hope the funding model for academic AI research can start looking more like biology and lab sciences rather than mathematics.12d
    Julius Adebayo@juliusadmlwell said.12d
    Amin Karbasi@aminkarbasi100%. This is the premise behind Foundation AI @fdtn_ai: we can do frontier research without training the next frontier model from scratch. There are deep, fundamental questions in how to extend these models, understand where they fail, and make them more useful. You don’t need 10,000 GPUs to do important science.12d
    Sreeram Kannan@sreeramkannanCo-signed. The era of high-frequency research. The following are common myths: 1. Humans are not needed to do research 2. Cost is so high that only frontier labs can do research. 3. Somehow ai based research is bad for research communities The most useful way to think of ai is **accelerating the timescale of research** It is similar to the transition from low-frequency research to high-frequency research akin to what happened in trading. What took years now takes days. Once it’s known that Euler has blowup, in the next week it was shown that Navier Stokes has blowup. The fluid science is nowhere near done, now, we need to figure out how to get to next steps: forcing removal for example. Compressing years to days and hours should be the goal. The @__alpoge__ and @DimitrisPapail of the world, who do fundamental work but are agent native, terminally online and collaborative will do very well. We need to build tools and technologies to help more scientists make this transition soon. We also need postAGI scientific institutions that operate at this high frequency: systems of collaboration, credit and funding need to adapt to this speed of light. It’s an extraordinarily exciting time. We should collaborate to solve humanity’s hardest problems at the speed of light.12d
    Alex Dimakis@AlexGDimakisDimitris has been proving you can do important AI basic research with a few GPUs and some agent max subscriptions. Some examples: ECHO paper, looped transformers, open mementos for context management come in mind.12d
    rohan anil@_arohan_What BigToken doesn’t want you to know is that they also read arxiv and X to get new ideas. Models are good at magnitude, direction is still hard. There is some uncertainty around for however long though, but its not today Please continue the amazing works like looping transformers and echo.12d
    Anastasios Nikolas Angelopoulos@ml_angelopoulosYep. This is a toxic (and crazy) outlook. There are plenty of important problems that basic research will solve outside of labs. Much of this pessimism btw is created by people that have never experienced excellence in pure research and thus don't have a frame of reference.12d
    Deborah Jacob@isthatdebbiejIdk who needs to hear this but being kind, chalant, and relentlessly curious is the real moat12d
    Phillip Isola@phillip_isola@DimitrisPapail I agree. Further, the basic research within frontier labs is mostly siloed there. If your goal is to further the world's understanding of intelligence, then I think basic research outside of frontier labs is still the best place to do that.12d
    Today's Rank

    —

    Not ranked yet

    Today's Rank

    —

    Not ranked yet