Mistaking fluent speech for fluent thought

A very common trope is to treat LLMs as if they were intelligent agents going out in the world and doing things. That’s just a category mistake. A much better way of thinking about them is as a technology that allows humans to access information from many other humans and use that information to make decisions. We have been doing this for as long a... See more

We, as humans, sometimes receive streams of tokens and produce tokens in response, forming words, sentences, lines of code ... but always with the ability to peek outside the stream and check in with ground-floor reality. We pause and consider: does this word really stand for the thing I intend it to stand for? Does this sentence capture the real e... See more