A rough analogy to the current LLM process is that making a new model is like baking a cake. You figure out your data and algorithms—like mixing the batter—and then you pretrain the model, that is, run it on a large number of computers for several months—like putting it in the oven—and then at the end you do some “post training”—like frosting and d... See more

LeCun points to four essential characteristics of human intelligence that current AI systems, including LLMs, can’t replicate: reasoning, planning, persistent memory, and understanding the physical world. He stresses that LLMs’ reliance on textual data severely limits their understanding of reality: “We’re easily fooled into thinking they are intel... See more

Even if LLMs don’t get smarter (though I suspect they will, and soon) new capabilities and modes of interacting with AIs, like agents and massive context windows, will help LLMs do dramatic new feats.

The need for better AI or LLM-specific infrastructure, along with the host of problems that come with non-deterministic of LLMs, means that there’s more software work ahead of us, not less. Abstraction layers like LLMs create more possibilities and thus, more work.

Is this a good thing or a bad thing? I’m not sure.

A great example of this is frontend... See more