there is a third way, the Goldilocks Zone : we get increasingly smart and capable helpers without eliminating the need for humans, or birthing beings that have any desire to eliminate humans.

That is the implication of agreeing with Leopold’s curve but disagreeing with the mapping. It’s what you get when you believe in straight lines on a graph... See more

We’re not even on the agency curve. It’s a different curve.

Look, I’m using knowledge and agency a little loosely. LLMs can do more than just regurgitate knowledge; even if you grant them reasoning, though, it still doesn’t feel like humanity emerges from LLM scaleup. They have no curiosity, and don’t ask their own questions.

Seven years ago, Kevin Kelly wrote, “Intelligence is not a single dimension, so ‘smarter than humans’ is a meaningless concept.” It bears repeating.

GPT-4 is a Smart High Schooler on one dimension, and a bottom centile High Schooler on others. It’s dangerous to assume that strengthening the strong dimension will strengthen the others. No A+ physics student ever became the star quarterback by becoming an A++ physics student.