Do Machine Learning Models Memorize or Generalize?
pair.withgoogle.com
Do Machine Learning Models Memorize or Generalize?
Differences between AI and human learning, focusing on batch training, lack of continual learning, and catastrophic forgetting in AI.
TRANSCRIPT
So mechanically, there seems to clearly be something different, or at least there's a big community of people in the computational neuroscience community that agree that there's got to be something different going on between these systems.
More abstractly, though, what's fundamentally different, which I think is maybe more practically
... See more
HOW IS THIS ALPHA EVEN PUBLIC? 10x SEARCH DEPTH VIA GRPO
The intuition has always been that scaling agentic search is a compute problem. It’s not. It’s a "stability-of-objective" problem. Most 8B models suffer from "horizon collapse" - they are mathematically "anxious" to terminate the search loop because their training... See more
Japanese martial arts traditions call these three stages of learning shu-ha-ri: first following rules faithfully (shu), then breaking from tradition (ha), and finally transcending formal structure altogether (ri). What makes this framework so revelatory for any AI doomsday discussion is how it illuminates the evolving relationship between
... See more