LLM problems observed in humans
Persistent hallucination
Understood in the medical sense, hallucination refers to when something appears to be real even as you know very well it isn’t. Having no direct insight into the “inner mental life” of models, we claim that every false fact they spit out is a form of hallucination. The meaning of the word is shifting from the medical sense... See more
Understood in the medical sense, hallucination refers to when something appears to be real even as you know very well it isn’t. Having no direct insight into the “inner mental life” of models, we claim that every false fact they spit out is a form of hallucination. The meaning of the word is shifting from the medical sense... See more
LLM problems observed in humans
Failure to generalize
By this point, it’s possible to explain what happens in a given situation, and watch the model apply the lessons learned to a similar situation. Not so with humans. When I point out that the same principles would apply elsewhere, their response will be somewhere along the spectrum of total bafflement on the one end and on the... See more
By this point, it’s possible to explain what happens in a given situation, and watch the model apply the lessons learned to a similar situation. Not so with humans. When I point out that the same principles would apply elsewhere, their response will be somewhere along the spectrum of total bafflement on the one end and on the... See more
LLM problems observed in humans
Small context window
The best thing about a good deep conversation is when the other person gets you: you explain a complicated situation you find yourself in, and find some resonance in their replies. That, at least, is what happens when chatting with the recent large models. But when subjecting the limited human mind to the same prompt—a rather... See more
The best thing about a good deep conversation is when the other person gets you: you explain a complicated situation you find yourself in, and find some resonance in their replies. That, at least, is what happens when chatting with the recent large models. But when subjecting the limited human mind to the same prompt—a rather... See more