Too Much Efficiency Makes Everything Worse: Overfitting and the Strong Version of Goodhart's Law

Stack rank all the candidates for a highly competitive school or job. Typically, offers would be made to the top-k candidates. Instead, make offers probabilistically, with probability proportional to

[approx # top tier candidates]

[candidate's stack rank]

. Benefits include: greater diversity of accepted candidates; less ridiculous resources spent b

... See more

Increasing efficiency is permeating almost every aspect of our society. If the thing that is being made more efficient is beneficial, then the increased efficiency makes the world a better place (overall, the world seems to be becoming a better place). If the thing that is being made more efficient is socially harmful, then the consequences of grea... See more

Increased efficiency can sometimes, counterintuitively, lead to worse outcomes. This is true almost everywhere.

Goodhart's law states that,  when a measure becomes a target, it ceases to be a good measure 2 . Goodhart proposed this in the context of monetary policy, but it applies far more broadly. In the context of overfitting in machine learning, it describes how the proxy objective we optimize ceases to be a good measure of the object

... See more

The strong version of Goodhart's law underlies most of my personal fears around AI (expect a future blog post about my AI fears!). If there is one thing AI will enable, it is greater efficiency, on almost all tasks, over a very short time period. We are going to need to simultaneously deal with massive numbers of diverse unwanted side effects,

... See more