Yann LeCun (@yannlecun)

I don’t wanna say “I told you so”, but I told you so. Quote: “Ilya Sutskever, co-founder of AI labs Safe Superintelligence (SSI) and OpenAI, told Reuters recently that results from scaling up pre-training - the phase of training an AI model that uses a vast amount of unlabeled data to understand language patterns and structures - have plateaued.”... See more

threads.net

RelatedInsightsHighlights

Thumbnail of www-x-com-johnrushx-status-1867723891688583356-87c83eed21244722

🚨Ilya Sutskever finally confirmed > scaling LLMs at the pre-training stage plateaued > the compute is scaling but data isn’t and new or synthetic data isn’t moving the needle What’s next > same as human brain, stopped growing in size but humanity kept advancing,... See more

John Rush

x.com

compute matters, but so does data. pre-training data wall -> post-training data boom @scale_AI is the data foundry that will accelerate us out of this rut 🫡

Alexandr Wang x.com

My point is, it's inevitable that China will catch up on the LLM front (whether through closed model like Bytedance or open source like Qwen DeepSeek etc.) So the next question becomes - what happens AFTER the models plateau? It comes down to atoms. It comes... See more

Zoomer x.com