Yann LeCun (@yannlecun)
threads.net
Yann LeCun (@yannlecun)
LeCun points to four essential characteristics of human intelligence that current AI systems, including LLMs, can’t replicate: reasoning, planning, persistent memory, and understanding the physical world. He stresses that LLMs’ reliance on textual data severely limits their understanding of reality: “We’re easily fooled into thinking they are intel... See more
Azeem Azhar • 🧠 AI’s $100bn question: The scaling ceiling
MargaretC added
Indeed, we may already be running into scaling limits in deep learning, perhaps already approaching a point of diminishing returns. In the last several months, research from DeepMind and elsewhere on models even larger than GPT-3 have shown that scaling starts to falter on some measures, such as toxicity, truthfulness, reasoning, and common sense
Gary Marcus • Deep Learning Is Hitting a Wall
Prashanth Narayan added
Sam Altmans Statement On AGI Is Bigger Than You Think!
youtube.com# Key Statements about AGI Development
## Leadership Predictions
### Sam Altman (OpenAI CEO)
- Predicts AGI arrival in 2025
- Claims the path to AGI is now clear
- States current hardware is sufficient for AGI development
- Believes superintelligence possible within "a few thousand days"
### Dario Amodei (Anthropic CEO)
- Predicts "powerful machine intelligence" by 2026
- Prefers not to use the term "AGI"
## Internal OpenAI Perspectives
### Noan Brown (OpenAI Researcher)
- Confirms Altman's statements align with median views of OpenAI researchers
- Supports timeline predictions
### Adam GPT (OpenAI GTM)
- Emphasizes Altman's precision in communications
- Notes public disconnect in understanding AI progress speed
## Technical Insights
### Scaling Laws
- Models become predictably smarter with more compute
- Both training and inference compute show consistent improvement patterns
- OpenAI plans to improve base models while scaling up inference compute
### Current Capabilities & Limitations
- Current models (like O1) show strong reasoning abilities
- Benchmark performance:
- Human baseline: 83.7%
- LLM baseline: ~40% on physical reasoning tasks
## Critical Perspectives
### Yan Lan (Turing Award Winner)
- Argues LLMs are insufficient for AGI
- States LLMs lack physical world understanding
- Questions current models' true reasoning capabilities
## Development Levels
1. Level 2 (Current): Reasoning systems
2. Level 3: Agent systems
3. Level 4: Innovators
- Transition between levels expected to be faster than initially thought
- Research shows LLMs can generate more novel ideas than human experts
## Research Developments
- Stanford study confirms LLMs can produce novel expert-level research ideas
- Recent breakthrough at OpenAI described as "breathtaking" (details undisclosed)
- O2 reportedly achieves 105% on certain GPA benchmarks
Prashanth Narayan and added
Some predict that with the dawn of AGI, machines that can improve themselves will trigger runaway growth in computer intelligence. Often called “the singularity,” or artificial superintelligence, this future involves computers whose ability to understand and manipulate the world dwarfs our own, comparable to the intelligence gap between human being
... See moreKai-Fu Lee • AI Superpowers: China, Silicon Valley, and the New World Order
How did we get to the doorstep of the next leap in prosperity?
In three words: deep learning worked.
In 15 words: deep learning worked, got predictably better with scale, and we dedicated increasing resources to it.
That’s really it; humanity discovered an algorithm that could really, truly learn any distribution of data (or really, the underlying “ru... See more
In three words: deep learning worked.
In 15 words: deep learning worked, got predictably better with scale, and we dedicated increasing resources to it.
That’s really it; humanity discovered an algorithm that could really, truly learn any distribution of data (or really, the underlying “ru... See more
samaltman.com • The Intelligence Age
How did we get to the doorstep of the next leap in prosperity?
In three words: deep learning worked.
In 15 words: deep learning worked, got predictably better with scale, and we dedicated increasing resources to it.
That’s really it; humanity discovered an algorithm that could really, truly learn any distribution of data (or really, the underlying “rules” that produce any distribution of data). To a shocking degree of precision, the more compute and data available, the better it gets at helping people solve hard problems. I find that no matter how much time I spend thinking about this, I can never really internalize how consequential it is.
Deep-learning systems are outstanding at interpolating between specific examples they have seen before, but frequently stumble when confronted with novelty.
Gary Marcus • Deep Learning Is Hitting a Wall
Prashanth Narayan added
In 2023, AI must absorb an influx of tourists – thin GPT wrappers, MBA tweet threads about AI trends, LinkedIn bios changing from #crypto to #AI. Genuine technologists will join the development effort, but filtering out negative human capital is challenging.
Undoubtedly, more builders are needed to apply and productionize the latest technologies,
... See moreJohn Luttig • Is AI the new crypto?
sari and added