I don't think that the scaling hypothesis gets recognized enough for how radical it is. For decades, AI sought some kind of master algorithm of intelligence. The scaling hypothesis says that there is none: intelligence is the ability to use more compute on more data.
Joscha Bachx.comI don't think that the scaling hypothesis gets recognized enough for how radical it is. For decades, AI sought some kind of master algorithm of intelligence. The scaling hypothesis says that there is none: intelligence is the ability to use more compute on more data.
The facts that (a) an AI somewhere could in principle do this task better, and (b) this task is no longer an economically rewarded element of a global economy, don’t seem to me to matter very much.
Dario Amodei • Machines of Loving Grace
So far, the (exponentially) more compute and data you put in, the more intelligence you get out. This effect is so clear and so important that I call the period since 2016 the scaling era of AI.55
Dwarkesh Patel • The Scaling Era: An Oral History of AI, 2019–2025
There are two paths to agents. When Sholto Douglas was on your podcast, he talked about scaling leading to more nines of reliability. That’s one path. The other path is the unhobbling path. The model needs to learn this System 2 process I described earlier. If it can learn that, it can use millions of tokens per query and think coherently.