Yann LeCun (@yannlecun)

I don’t wanna say “I told you so”, but I told you so. Quote: “Ilya Sutskever, co-founder of AI labs Safe Superintelligence (SSI) and OpenAI, told Reuters recently that results from scaling up pre-training - the phase of training an AI model that uses a vast amount of unlabeled data to understand language patterns and structures - have plateaued.”... See more

threads.net

RelatedInsightsHighlights

Ilya Sutskever, perhaps the most influential proponent of the AI "scaling hypothesis," just told Reuters that scaling has plateaued. This is a big deal! This comes on the heels of a big report that OpenAI's in-development Orion model had disappointing results. 🧵 https://t.co/DiRb9sOHt8

Garrison Lovely

x.com

🚨Ilya Sutskever finally confirmed > scaling LLMs at the pre-training stage plateaued > the compute is scaling but data isn’t and new or synthetic data isn’t moving the needle What’s next > same as human brain, stopped growing in size but humanity kept advancing,... See more

John Rush

x.com

Ilya Sutskever – We're moving from the age of scaling to the age of research

youtube.com