AI

I don't think that the scaling hypothesis gets recognized enough for how radical it is. For decades, AI sought some kind of master algorithm of intelligence. The scaling hypothesis says that there is none: intelligence is the ability to use more compute on more data.

A few years before quantum physics was discovered, its core ideas were completely outside of human discourse, thoughts, and no amount of circling the same box (which is what reasoning models do) would get us there.

LLMs, says Chollet, irrespective of their size, are limited in their ability to solve problems that require recombining what they have learnt to tackle new tasks. “LLMs cannot truly adapt to novelty because they have no ability to basically take their knowledge and then do a fairly sophisticated recombination of that knowledge on the fly to adapt... See more