Skip to content
Rabin Poudyal
Rabin Poudyal
← All models

Economics and incentives

Goodhart's law

When a measure becomes a target it stops being a good measure, because people optimise the number instead of the thing it stood for.

Charles Goodhart, in a 1975 paper for a Reserve Bank of Australia conference on British monetary management. The short form everyone quotes is Marilyn Strathern's, from a 1997 paper on audit in British universities.

Goodhart's original wording was technical: "Any observed statistical regularity will tend to collapse once pressure is placed upon it for control purposes." He was writing about monetary policy, where relationships between measures of money and the economy that had looked stable broke down once the authorities used them as targets. Strathern, an anthropologist writing about audit in British universities, gave it the form that spread: "When a measure becomes a target, it ceases to be a good measure." The common reading is that a proxy is informative only while nobody is being judged by it. The moment the proxy pays, people produce the proxy.

It holds because a metric is a compressed signal. It tracked the thing you cared about because, in the past, the cheapest way to move the metric was to do the real thing. Attach a reward and you change what the cheapest way is: reclassify the case, pick the easy patients, teach to the test, close the ticket without fixing the bug. The correlation that justified the metric was conditional on nobody gaming it, and the target removes that condition. In practice you see it in call-centre handling times, exam scores, lines of code, citation counts and, as the example shows, in a bank's cross-selling numbers.

Not every metric collapses. The law bites when the people measured can move the number cheaply and the real outcome is hard to see; a metric that is close to the thing itself, such as revenue or uptime as users experience it, resists better. The answer is not to stop measuring but to use several measures, keep some unannounced, rotate them, and treat a metric that improves while complaints rise as a warning. On credit: Goodhart stated the collapse, Strathern wrote the sentence, and the two are often swapped. Second-order thinking is the habit that catches this before launch: ask how people will respond to the target, not just what the target says.

Example

On 8 September 2016 the US Consumer Financial Protection Bureau fined Wells Fargo $100 million, part of $185 million in penalties with the Office of the Comptroller of the Currency and the City of Los Angeles. The bank had set sales targets and paid bonuses for signing existing customers up to new products. To hit the targets, employees had opened roughly 1.5 million deposit accounts and about 565,000 credit card accounts that customers may not have authorised. The cross-selling metric looked excellent right up to the fine.

Where it applies

Any benchmark you optimise against becomes less informative as you optimise: a model tuned to score well on an evaluation set learns the evaluation set. Keep a held-out set that nobody tunes on, and read real transcripts. In teams, treat velocity, test coverage and ticket counts as thermometers, not as goals.

Think about it

Which number on your dashboard would you stop trusting if everyone knew their bonus depended on it?

Applied it to something real? Log it.

Test yourselfA metric that has predicted quality well is made the team's target. What does Goodhart's law predict, and why?

It stops predicting quality. The correlation held while the cheapest way to move the number was doing the real work; once the number pays, cheaper ways of moving it appear.

Related models

Sources and further reading