Too Much Efficiency Makes Everything Worse: Overfitting and the Strong Version of Goodhart's Law
Jascha Sohl-Dicksteinsohl-dickstein.github.ioSaved by sari and
Too Much Efficiency Makes Everything Worse: Overfitting and the Strong Version of Goodhart's Law
HOW IS THIS ALPHA EVEN PUBLIC? 10x SEARCH DEPTH VIA GRPO
The intuition has always been that scaling agentic search is a compute problem. It’s not. It’s a "stability-of-objective" problem. Most 8B models suffer from "horizon collapse" - they are mathematically "anxious" to terminate the search loop because their training... See more
Goodhart's law states that, when a measure becomes a target, it ceases to be a good measure 2 . Goodhart proposed this in the context of monetary policy, but it applies far more broadly. In the context of overfitting in machine learning, it describes how the proxy objective we optimize ceases to be a good measure of the objective we care about.