🧠AI’s $100bn question: The scaling ceiling
LeCun points to four essential characteristics of human intelligence that current AI systems, including LLMs, can’t replicate: reasoning, planning, persistent memory, and understanding the physical world. He stresses that LLMs’ reliance on textual data severely limits their understanding of reality: “We’re easily fooled into thinking they are intel... See more
Azeem Azhar • 🧠AI’s $100bn question: The scaling ceiling
The reason experts are so split on whether or not we're on an exponential path to AI is that intelligence is multi-dimensional. You can be both exponentially exploding/scaling in one dimension and stagnating in another. You can be both superhuman and subhuman at the same time. People tend to focus on the dimensions most salient to their inclination... See more
No physical process can grow indefinitely.
There are always friction terms in the dynamics equation that eventually become dominant (energy consumption, heat dissipation, quantum effects, thermal fluctuations, communication bandwidth, mass/energy density....).
Even processes that *appear* exponential on a long time scale are actually a succession of... See more
There are always friction terms in the dynamics equation that eventually become dominant (energy consumption, heat dissipation, quantum effects, thermal fluctuations, communication bandwidth, mass/energy density....).
Even processes that *appear* exponential on a long time scale are actually a succession of... See more
Yann LeCun • Tweet
Working in technology means one thing above all else: chasing scale. There is a reason why much of the tech world is obsessed with growth. Free from physical constraints, digital systems can scale to an incomprehensible size. The appeal of conquering the engineering, design and business challenges of mega-scale is strong, the rewards immense. But u
... See more