r/MachineLearning - Reddit
reddit.com
r/MachineLearning - Reddit
Rather than showing the capability for generalized logical inference, [reasoning] chain-of-thought models are "a sophisticated form of structured pattern matching" that "degrades significantly" when pushed even slightly outside of its training distribution.
-Is Chain-of-Thought Reasoning of LLMs a Mirage? A Data Distribution Lens
In addition to insider bullishness, I think there’s a strong intuitive case for why it should be possible to find ways to train models with much better sample efficiency (algorithmic improvements that let them learn more from limited data). Consider how you or I would learn from a really dense math textbook:
- What a modern LLM does during training i
SITUATIONAL AWARENESS - The Decade Ahead • I. From GPT-4 to AGI: Counting the OOMs
“Let’s use an LLM for that”
One of the most deteriorating trend I am seeing right now.
Carefully crafting a problem, designing a model specific to that and then optimizing for number of parameters etc are relics of the past.
MNIST classification? LLMs___LINEBREAK... See more