scientific research

If you have a theory that’s been working pretty well for a while — it made good predictions, it solved real problems, it explained a lot of mysteries — you should stick with it in the face of apparent contradictions, at least for a while. When you hit a snag with a reliable theory, think “maybe it’s complicated” instead of “oh it’s wrong”. It may

... See more

Through extensive experimentation across diverse puzzles, we show that frontier LRMs face a complete accuracy collapse beyond certain complexities. Moreover, they exhibit a counter-intuitive scaling limit: their reasoning effort increases with problem complexity up to a point, then declines despite having an adequate token budget. By comparing LRMs

... See more

1. We asked students to test a specific hypothesis with a dataset: Is BMI related to number of steps? Other students were just asked to analyze the dataset. The students with a specific hypothesis were 3X less likely to see the gorilla in the data! https://genomebiology.biomedcentral.com/articles/10.1186/s13059-020-02133-w…