Sublime
An inspiration engine for ideas
a quick note from @karpathy talk, which afterwards I was fully immersed in the talk I stopped writing down/taking notes:
While it is interesting to think about LLMs in terms of "electricity" -> CAPEX to train an LLM (~= to build the grid) -> OPEX to serve intelligence over increasingly homogeneous API (prompt, image,... See more
Michael Jordan gave a short, excellent, and provocative talk recently in Paris - here's a few key ideas
- It's all just machine learning (ML) - the AI moniker is hype
- The late Dave Rumelhart should've received a Nobel prize for his early ideas on making backprop... See more
Every time someone proposes training a model for some domain (virtual cells/materials/whatever), the first sentence should be: input data: x, output data: y, loss function: z.
Would make it a lot easier for dum dums like me to understand what exactly is being proposed.
Dwarkesh Patelx.comNice article in Financial Time where I explain that Auto-Regressive LLM are insufficient to reach human-level intelligence (or even cat-level intelligence).
But alternative architectures that I call "objective driven" may reach human-level intelligence one day.
They use world models based on JEPA (Joint... See more
Yann LeCunx.comLots of confusion about what a world model is. Here is my definition:
Given:
- an observation x(t)
- a previous estimate of the state of the world s(t)
- an action proposal a(t)
- a latent variable proposal z(t)
A world model... See more
Yann LeCunx.com
