But should we be scared in the sense that we think we’re doomed? No. You’re coming up with an incredible amount of leverage in terms of how the AIs will interact with the world, how they’re trained, and the default values they start with.

One of the first things [OpenAI cofounder and former chief scientist] Ilya Sutskever said to me was, “Look. The models just want to learn. You have to understand this. The models just want to learn.” It was a bit like a Zen kōan. I listened to this and I became enlightened.

A traditional Stockfish or Deep Blue system would look at millions of possible moves for every decision. AlphaZero and AlphaGo may look at tens of thousands of possible positions in order to make a decision about what to move next. A human grandmaster or world champion probably only looks at a few hundred moves in order to make their very good

It doesn’t seem particularly compelling. One source of evidence is work by Suzana Herculano-Houzel, a neuroscientist who has dissolved the brains of many creatures to determine how many neurons are present. She’s found a lot of interesting scaling laws. She has a paper discussing the human brain as a scaled-up primate brain.60 Across a wide variety

model, you get this raw bundle of capabilities. That’s useful. The unhobbling from GPT-2 to GPT-4 took this raw mass and RLHF’d

This pretraining advantage is also the difference between LLMs and robotics. People used to say the slow progress in robotics was a hardware problem. The hardware issue is getting solved, but you still don’t have this huge advantage of bootstrapping with pretraining. You don’t have all this unsupervised learning you can do. You have to start right

The

work, we’re left with the baseline—the economy growing at only 2 percent a year, so we have only 2 percent more resources a year to spend on AI. You’re talking about decades, then, before you can train a $10 trillion model.

Richard Sutton’s “Bitter Lesson” essay64 says that there are two things you can scale: search and learning.