AI Revolution - Transformers and Large Language Models (LLMs)

To the extent that large language models (and I should note that while I’m focusing on image generation, there are a whole host of companies working on text output as well) are dependent not on carefully curated data, but rather on the Internet itself, is the extent to which AI will be democratized, for better or worse.

Interface > Data > Model: The “interface” and “data” layers will further distinguish market leaders while the “models” layer becomes increasingly commoditized and pushed to the edge. As a growing number of our everyday use-cases of powerful GenerativeAI models fall below the frontier of “the best models,” they will be enabled by cheaper commoditize... See more

LeCun points to four essential characteristics of human intelligence that current AI systems, including LLMs, can’t replicate: reasoning, planning, persistent memory, and understanding the physical world. He stresses that LLMs’ reliance on textual data severely limits their understanding of reality: “We’re easily fooled into thinking they are intel... See more

ModelsText is the most advanced domain. However, natural language is hard to get right, and quality matters. Today, the models are decently good at generic short/medium-form writing (but even so, they are typically used for iteration or first drafts). Over time, as the models get better, we should expect to see higher quality outputs, longer-form c... See more