🧠 AI’s $100bn question: The scaling ceiling

First, operationalisation takes a long time, often revealing hidden potential in existing technologies. As Jack Clark notes, “if we stopped all AI progress today, there’s a huge capability overhang”. Even without further model development, building the right scaffolding can unlock surprising capabilities. This scaffolding isn’t just software; it in... See more

One way of thinking about this is Daniel Kahneman’s simple model of thinking: System 1 and System 2. System 1 thinking is fast and intuitive. Current AI models’ pattern recognition and next-token prediction are good examples of this. System 2 thinking is slow and analytical, akin to genuine reasoning and understanding. It is System 2 thinking where... See more

The incentive to find breakthrough science that provides a performance pathway other than scaling is increasing. A GPT-6 class model will cost $10 billion and three years to improve on GPT-5 by some uncertain degree. That’s a ton of time and a lot of cash for an uncertain payout: in other words, a substantial prize for anyone who can figure out pro... See more

LeCun points to four essential characteristics of human intelligence that current AI systems, including LLMs, can’t replicate: reasoning, planning, persistent memory, and understanding the physical world. He stresses that LLMs’ reliance on textual data severely limits their understanding of reality: “We’re easily fooled into thinking they are intel... See more

o, alternative pathways to building Type-2 reasoning-capable AI systems, likely using neurosymbolic approaches, have become much more attractive. People like Gary Marcus have argued for neurosymbolic approaches for decades. Such approaches combine the pattern recognition of neural nets, like LLMs, with symbolic reasoning’s logic and rules. Vinod Kh... See more