Reasoning skills of large language models are often overestimated
ChatGPT is a Knowledgeable but Inexperienced Solver: An Investigation of Commonsense Problem in Large Language Models
https://t.co/0vCvHIFWTl... See more
What this means, in part, is that LLMs never know a fact or understand a concept in the way that we do. Instead, every time you prompt an LLM with a question, or ask it to take some action, you are simply asking it to make a prediction about what tokens are most likely to follow the tokens that comprise your prompt in a contextually relevant way. A
... See moreGreg Beato • Superagency
Rather than showing the capability for generalized logical inference, [reasoning] chain-of-thought models are "a sophisticated form of structured pattern matching" that "degrades significantly" when pushed even slightly outside of its training distribution.
-Is Chain-of-Thought Reasoning of LLMs a Mirage? A Data Distribution Lens