I don't think that the scaling hypothesis gets recognized enough for how radical it is. For decades, AI sought some kind of master algorithm of intelligence. The scaling hypothesis says that there is none: intelligence is the ability to use more compute on more data.
I don't think that the scaling hypothesis gets recognized enough for how radical it is. For decades, AI sought some kind of master algorithm of intelligence. The scaling hypothesis says that there is none: intelligence is the ability to use more compute on more data.
Indeed, we may already be running into scaling limits in deep learning, perhaps already approaching a point of diminishing returns. In the last several months, research from DeepMind and elsewhere on models even larger than GPT-3 have shown that scaling starts to falter on some measures, such as toxicity, truthfulness, reasoning, and common sense
Gary Marcus • Deep Learning Is Hitting a Wall
As we successfully apply simpler, narrow versions of intelligence that benefit from faster computers and lots of data, we are not making incremental progress, but rather picking low-hanging fruit. The jump to general “common sense” is completely different, and there’s no known path from the one to the other.
Erik J. Larson • The Myth of Artificial Intelligence: Why Computers Can’t Think the Way We Do
“AI being smarter than a human is impossible because there are a bunch of ways to be intelligent and no particularly special way to be generally intelligent, i.e., it’s not a scale.”
Erik Hoel • How to navigate the AI apocalypse as a sane person
I still think that we are thinking about scaling of network-based intelligence looking at this particular network. An extended network of machines and humans in larger networks facilitated by smaller and increasingly embodied devices might in themselves add an AI-augmented neocortex to our (human-machine) systems. Intelligence will continue to emer... See more
Azeem Azhar • 🧠 AI’s $100bn question: The scaling ceiling
The last 200 years have seen the influence of mathematics deepen across almost all domains of human activity, amply supported by torrents of data and dramatic increases in computing power. Now we use math to model nuclear wars, pick players for baseball teams, track changes in literature, and forecast presidential elections. Sometimes, it seems the... See more
Hannes Malmberg • How mathematics built the modern world - Works in Progress
Suppose non-conscious algorithms could eventually outperform conscious intelligence in all known data-processing tasks – what, if anything, would be lost by replacing conscious intelligence with superior non-conscious algorithms?