
Saved by Aman and
Goodhart's law
Saved by Aman and
THE TRAP: SEEKING THE WRONG GOAL System behavior is particularly sensitive to the goals of feedback loops. If the goals—the indicators of satisfaction of the rules—are defined inaccurately or incompletely, the system may obediently work to produce a result that is not really intended or wanted. THE WAY OUT Specify indicators and goals that reflect
... See moreLots of us have heard about the tragedy of the commons: When people share a resource but don’t own it, everything goes to hell because they don’t care.
good numbers don't mean good impact. We can hit every target and still harm the system we operate in.