more difficult. One is that as these models get smarter, they’re going

70 When comparing RLHF versus non-RLHF models, RLHF is equivalent to increasing the model size 100 times in terms of the resulting increase in human evaluators’ preference ratings.

model, you get this raw bundle of capabilities. That’s useful. The unhobbling from GPT-2 to GPT-4 took this raw mass and RLHF’d

There are two paths to agents. When Sholto Douglas was on your podcast, he talked about scaling leading to more nines of reliability. That’s one path. The other path is the unhobbling path. The model needs to learn this System 2 process I described earlier. If it can learn that, it can use millions of tokens per query and think coherently.

GPT-4 instead thinks for a few hundred tokens. It’s equivalent to me thinking for three minutes.

Right now, GPT-4 can do a few hundred tokens of chain-of-thought prompting.

The human brain has 30 to 300 trillion synapses. It’s obviously not a 1-to-1 mapping between machine parameters and animal synapses, and we can debate these numbers, but it seems pretty plausible that we’re still below the scale of the human brain.

claims that GPT-4 has around 1 trillion parameters… The human brain has 30 to 300 trillion synapses. It’s obviously not a 1-to-1 mapping between machine parameters and animal synapses, and we can debate these numbers, but it seems pretty plausible that we’re still below the scale of the human brain.

don’t want people to come away thinking that models aren’t going to get much better. The jumps we’ve seen so far are huge. Even if those continue on a smaller scale, we’re still in for extremely smart, very reliable agents over the next couple of orders of magnitude. We have a lot more jumps coming. Even if those jumps are smaller, relatively speak