The Scaling Era: An Oral History of AI, 2019–2025

But should we be scared in the sense that we think we’re doomed? No. You’re coming up with an incredible amount of leverage in terms of how the AIs will interact with the world, how they’re trained, and the default values they start with.

I don’t know what they’re selling it for. Suppose you sold some AI add-on for $100 a month to a third of Microsoft Office subscribers—that’d be $100 billion right there.

Data centers currently account for 4 percent of US power generation. According to a 2024 estimate by the Electric Power Research Institute, this will rise to 9 percent by 2030, although this includes non-AI centers

One of the first things [OpenAI cofounder and former chief scientist] Ilya Sutskever said to me was, “Look. The models just want to learn. You have to understand this. The models just want to learn.” It was a bit like a Zen kōan. I listened to this and I became enlightened.

you look at the harms that people try to do through social networks, some are not very adaptively adversarial. For example, hate speech. People aren’t getting better at being racist.

You talk about the enormous power that superintelligence and the government will have. It’s pretty plausible that in the alternative world, one AI company will have that power. Say OpenAI has a six-month lead. You’re talking about the most powerful weapon ever. So you’re making a radical bet on a private company CEO as the benevolent dictator.

A traditional Stockfish or Deep Blue system would look at millions of possible moves for every decision. AlphaZero and AlphaGo may look at tens of thousands of possible positions in order to make a decision about what to move next. A human grandmaster or world champion probably only looks at a few hundred moves in order to make their very good deci

The need to consider the cumulative probability of success lowers my estimated probability of AGI by 2040 down to 60 percent.