Talking about large language models

A very common trope is to treat LLMs as if they were intelligent agents going out in the world and doing things. That’s just a category mistake. A much better way of thinking about them is as a technology that allows humans to access information from many other humans and use that information to make decisions. We have been doing this for as long a... See more

We’re far from replacing humans for many tasks now being delegated to large language models. That doesn’t mean LLMs can’t be helpful; it just means they’re being used wrong. The key is applying them to tasks they’re well-suited to, such as analyzing, synthesizing, and manipulating data, and not for making important decisions on behalf of people.

An ... See more

I have long been a subscriber to the idea that any intelligence needs to be embodied and situated. The very first problems a baby has to solve are, How do I move my body? How do I move around in the world? Those tasks are entirely missing from existing large language models. And, as many people noted, the phrase “large language model” is a misnomer... See more

Pre-trained language models (LM), by contrast, are capable of directly generating prose that may be responsive to an information need, but at present they are *dilettantes* rather than domain experts – they do not have a true understanding of the world, they are prone to hallucinating, and crucially they are incapable of justifying their utterances... See more

LeCun points to four essential characteristics of human intelligence that current AI systems, including LLMs, can’t replicate: reasoning, planning, persistent memory, and understanding the physical world. He stresses that LLMs’ reliance on textual data severely limits their understanding of reality: “We’re easily fooled into thinking they are intel... See more

Language Models Represent Space and Time , Wes Gurnee and Max Tegmark argue that “modern LLMs acquire structured knowledge about fundamental dimensions such as space and time, supporting the view that they learn not merely superficial statistics, but literal world models”, based on some analyses of the alleged capacity of LLMs to understand geograp... See more