LeCun points to four essential characteristics of human intelligence that current AI systems, including LLMs, can’t replicate: reasoning, planning, persistent memory, and understanding the physical world. He stresses that LLMs’ reliance on textual data severely limits their understanding of reality: “We’re easily fooled into thinking they are... See more

So one way to make LLMs more goal-directed is just to tell them to think harder. But the limitation of CoT prompting is that it assumes that the model can effortlessly figure out how best to break down the problem into manageable steps. In fact, many real-world problems have several competing solutions, some of which may turn out to be dead ends.

... See more

LLMs are extremely good at a type of knowledge called tacit knowledge (knowledge about something as it pertains to something else) but are extremely poor at pretty much any other type as far as I can see. It just so happens that tacit knowledge also happens to be the type of knowledge that drives natural language so it makes them looks super smart,... See more