scientific research

Through extensive experimentation across diverse puzzles, we show that frontier LRMs face a complete accuracy collapse beyond certain complexities. Moreover, they exhibit a counter-intuitive scaling limit: their reasoning effort increases with problem complexity up to a point, then declines despite having an adequate token budget. By comparing LRMs

... See more

Things change as contradictory evidence piles up, but even then, it doesn’t mean you should scrap the theory you started out with. Everyone back in the 1870s made a big mistake throwing out their perfectly good “disease of deficiency” theory as soon as there were a few contradictory stories from polar explorers. Their mistake was thinking “maybe

... See more

Real explanations will sometimes sound weird, crazy, or too complicated because reality itself is often weird, crazy, or too complicated. It’s unfortunate, but scurvy is really the BEST CASE SCENARIO. The answer ended up being almost comically simple: it’s just a disease of deficiency, eat one of these foods containing this vitamin and be instantly

... See more

If you have a theory that’s been working pretty well for a while — it made good predictions, it solved real problems, it explained a lot of mysteries — you should stick with it in the face of apparent contradictions, at least for a while. When you hit a snag with a reliable theory, think “maybe it’s complicated” instead of “oh it’s wrong”. It may

... See more

Lots of theories have been tried, and lots of them have been given up because of something that looks like contradictory evidence. But the evidence might not actually be a contradiction — the real explanation might just be slightly more complicated than people realized. Go back and revisit scientific near-misses, maybe there’s a wrinkle they didn’t

... See more