Yann LeCun (@yannlecun)

I don’t wanna say “I told you so”, but I told you so. Quote: “Ilya Sutskever, co-founder of AI labs Safe Superintelligence (SSI) and OpenAI, told Reuters recently that results from scaling up pre-training - the phase of training an AI model that uses a vast amount of unlabeled data to understand language patterns and structures - have plateaued.” .... See more

threads.net

RelatedHighlights

Whereas if scaling was true, you would automatically get much more impressive results than GPT-2, guaranteed. I opened up the second page [of the paper] and I saw the few-shot learning chart, and I’m like, “Holy shit, we live in the scaling world. Legg and Moravec and Kurzweil were right!”

Dwarkesh Patel • The Scaling Era: An Oral History of AI, 2019–2025

Yann LeCun: I'm not interested in LLMs anymore - they're the past. The future is in four more interesting areas: machines that understand the physical world, persistent memory, reasoning, and planning. https://t.co/qRH7XVlbWq

Victor x.com

In 2020, Jared Kaplan and his collaborators at OpenAI suggested that there was a set of “scaling laws” for neural network models of language; they found that the more data they fed into their neural networks, the better those networks performed.10 The implication was that we could do better and better AI if we gather more data and apply deep learni... See more

Dwarkesh Patel • The Scaling Era: An Oral History of AI, 2019–2025

Gary Marcus • Deep Learning Is Hitting a Wall