Saved by Packy McCormick and
The Bitter Lesson
I've just drafted a new blogpost
"The Bitter Lesson's Bitter Lesson"
Richard Sutton and Dwarkesh discussed the Bitter Lesson, where Richard argued that babies and animals don’t learn through imitation, so state-of-the-art LLMs are pursuing the wrong path by imitating humans through... See more
The Bitter Lesson versus The Garbage Can
oneusefulthing.org
Finally had a chance to listen through this pod with Sutton, which was interesting and amusing.
As background, Sutton's "The Bitter Lesson" has become a bit of biblical text in frontier LLM circles. Researchers routinely talk about and ask whether this or that approach or idea is sufficiently "bitter lesson pilled"... See more
Andrej Karpathyx.com