Yann LeCun (@yannlecun)

I don’t wanna say “I told you so”, but I told you so. Quote: “Ilya Sutskever, co-founder of AI labs Safe Superintelligence (SSI) and OpenAI, told Reuters recently that results from scaling up pre-training - the phase of training an AI model that uses a vast amount of unlabeled data to understand language patterns and structures - have plateaued.”... See more

threads.net

RelatedInsightsHighlights

Thumbnail of www-x-com-johnrushx-status-1867723891688583356-87c83eed21244722

🚨Ilya Sutskever finally confirmed > scaling LLMs at the pre-training stage plateaued > the compute is scaling but data isn’t and new or synthetic data isn’t moving the needle What’s next > same as human brain, stopped growing in size but humanity kept advancing,... See more

John Rush

x.com

Ex-OpenAI researcher (now at META) explains how scaling laws can never fail because it’s a reflection of the data structure “what really failed is the data” OpenAI has a skill issue. https://t.co/YZyKygsfhB

NIK x.com

LeCun points to four essential characteristics of human intelligence that current AI systems, including LLMs, can’t replicate: reasoning, planning, persistent memory, and understanding the physical world. He stresses that LLMs’ reliance on textual data severely limits their understanding of reality: “We’re easily fooled into thinking they are... See more

Azeem Azhar • 🧠 AI’s $100bn question: The scaling ceiling