Goodhart's law explained

Goodhart's law should not be confused with Godwin's law.

Goodhart's law is an adage often stated as, "When a measure becomes a target, it ceases to be a good measure". It is named after British economist Charles Goodhart, who is credited with expressing the core idea of the adage in a 1975 article on monetary policy in the United Kingdom:^[1]

It was used to criticize the British Thatcher government for trying to conduct monetary policy on the basis of targets for broad and narrow money,^[2] but the law reflects a much more general phenomenon.^[3]

Priority and background

Numerous concepts are related to this idea, at least one of which predates Goodhart's statement.^[4] Notably, Campbell's law likely has precedence, as Jeff Rodamar has argued, since various formulations date to 1969.^[5] Other academics had similar insights at the time. Jerome Ravetz's 1971 book Scientific Knowledge and Its Social Problems^[6] also predates Goodhart, though it does not formulate the same law. He discusses how systems in general can be gamed, focuses on cases where the goals of a task are complex, sophisticated, or subtle. In such cases, the persons possessing the skills to execute the tasks properly seek their own goals to the detriment of the assigned tasks. When the goals are instantiated as metrics, this could be seen as equivalent to Goodhart and Campbell's claim.

Shortly after Goodhart's publication, others suggested closely related ideas, including the Lucas critique (1976). As applied in economics, the law is also implicit in the idea of rational expectations, a theory in economics that states that those who are aware of a system of rewards and punishments will optimize their actions within that system to achieve their desired results. For example, if an employee is rewarded by the number of cars sold each month, they will try to sell more cars, even at a loss.

While it originated in the context of market responses, the law has profound implications for the selection of high-level targets in organizations. Jon Danielsson states the law as

And suggested a corollary for use in financial risk modelling:

Mario Biagioli related the concept to consequences of using citation impact measures to estimate the importance of scientific publications:^{[7] [8]}

The law is illustrated in the 2018 book The Tyranny of Metrics by Jerry Z. Muller.^[9]

Generalization

Later writers generalized Goodhart's point about monetary policy into a more general adage about measures and targets in accounting and evaluation systems. In a book chapter published in 1996, Keith Hoskin wrote:

In a 1997 paper responding to the work of Hoskin and others on financial accounting and grades in education, anthropologist Marilyn Strathern expressed Goodhart's Law as "When a measure becomes a target, it ceases to be a good measure," and linked the sentiment to the history of accounting stretching back into Britain in the 1800s:

Examples

• . The statement denounces several problems in science and as Goodhart's law explains, one of them is that measurement has become a target. The correlation between h-index and scientific awards is decreasing since widespread usage of h-index.^[10]
• International Union for Conservation of Nature's measure of extinction can be used to remove environmental protections, which resulted in IUCN becoming more conservative in labeling something as extinct.^{[11] [12]}
• According to How to Read Numbers, the law applied to the British government response to the COVID-19 pandemic when it announced a target of 100,000 COVID-19 tests per day—initially a target for tests actually carried out and later for maximum capacity of test-taking. The number of useful diagnostic tests was far lower than the government-reported number when it announced it had met the target.^[13]
• In the HBO television show, The Wire, Roland 'Prezbo' Pryzbylewski draws parallels between public school policy of teaching to the test and improving crime statistics through reclassification of those crimes as "juking the stats." He notes that they both are examples of strategies that are ineffective at substantive improvements in learning and public safety respectively because they cater to ineffective metrics of standardized test scores and reported crime rates.^[14]

See also

• – "The more any quantitative social indicator is used for social decision-making, the more subject it will be to corruption pressures"
• - when incentives designed to solve a problem end up rewarding people for making it worse
• - the tendency to search for and recall information that confirms or supports one's prior beliefs
• - when people modify an aspect of their behavior in response to their awareness of being observed
• - manipulating rules and procedures to obtain a desired outcome
• – it is naive to try to predict the effects of a change in economic policy entirely on the basis of relationships observed in historical data
• – involves making a decision based solely on quantitative observations (or metrics) and ignoring all others
• - tendency for decision-makers to place excessive emphasis on selected measurable goals
• - an analysis that corresponds too closely or exactly to a particular set of data
• – individuals are promoted based on success in their previous roles, and not the role of the new position
• - circular relationships between cause and effect
• - when an abstract belief is treated as if it were a real thing
• — sometimes referred to via the quote "the map is not the territory". It is a type of reification (fallacy), wherein a model of something is treated as if it were exactly like the actual thing being modeled. Goodhart's law addresses a subset of map-territory problems.
  • - in business, when a measure of a construct of interest evolves to replace that construct

Further reading

• Web site: Chrystal. K. Alec. Mizen. Paul D.. 12 November 2001. Goodhart's Law: Its Origins, Meaning and Implications for Monetary Policy. 3 July 2020.
• Web site: Manheim . David . 9 June 2016 . Goodhart's Law and Why Measurement is Hard . 3 July 2020 . ribbonfarm . en-US.
• The Laws Of The Office. Planet Money. NPR. Malone. Kenny. Gonzalez. Sarah. Horowitz-Ghazi. Alexi. Goldmark. Alex. 21 November 2018. 3 July 2020. en-us.

Notes and References

1. Encyclopedia: Problems of Monetary Management: The U.K. Experience . Papers in Monetary Economics . Reserve Bank of Australia . Sydney . Goodhart . Charles . Papers in monetary economics 1975; 1; 1. - [Sydney]. - 1975, p. 1-20 . 1975 . Charles Goodhart . 1.
2. Book: Smith . David . The Rise And Fall of Monetarism . 1987 . . 9780140227543 . London.
3. 1803.04585. cs.AI. David. Manheim. Scott. Garrabrant. Categorizing Variants of Goodhart's Law. 2018.
4. Web site: Manheim . David . 29 September 2016 . Overpowered Metrics Eat Underspecified Goals . 26 January 2017 . ribbonfarm . en-US.
5. Rodamar . Jeffery . 28 November 2018 . There ought to be a law! Campbell versus Goodhart . Significance . 15 . 6 . 9 . 10.1111/j.1740-9713.2018.01205.x . free.
6. Book: Ravetz, Jerome R. . Scientific knowledge and its social problems . Transaction Publishers . 1971 . 1-56000-851-2 . New Brunswick, New Jersey . 295–296 . 32779931 .
7. Biagioli . Mario . 12 July 2016 . Watch out for cheats in citation game . Nature . 535 . 7611 . 201 . 2016Natur.535..201B . 10.1038/535201a . 27411599. free .
8. Varela . Diego . Benedetto . Giacomo . Sanchez-Santos . Jose Manuel . Editorial statement: Lessons from Goodhart's law for the management of the journal . European Journal of Government and Economics . 30 December 2014 . 3 . 2 . 100–103 . 10.17979/ejge.2014.3.2.4299. 152551763 . 8 February 2022. free . 2183/23376 . free .
9. Book: Muller, Jerry Z. . The Tyranny of Metrics . 2018 . Princeton University Press . 978-0-691-19126-3.
10. Koltun . V . Hafner . D . The h-index is no longer an effective correlate of scientific reputation. . PLOS One . 2021 . 16 . 6 . e0253397 . 10.1371/journal.pone.0253397 . 34181681 . 8238192 . 2102.03234 . 2021PLoSO..1653397K . Our results suggest that the use of the h-index in ranking scientists should be reconsidered, and that fractional allocation measures such as h-frac provide more robust alternatives. . free . Companion webpage
11. Web site: Mooers . Arne . 2022-05-23 . When is a species really extinct? . 2023-06-23 . The Conversation . en.
12. Martin . T. E. . Bennett . G. C. . Fairbairn . A. . Mooers . A. O. . March 2023 . 'Lost' taxa and their conservation implications . Animal Conservation . en . 26 . 1 . 14–24 . 10.1111/acv.12788 . 2023AnCon..26...14M . 248846699 . 1367-9430.
13. Book: How to Read Numbers. Weidenfeld & Nicolson. 2021. Chivers. Tom. Chivers. David. 9781474619974. 22: Goodhart's Law.
14. Web site: Roshan . Revanka . Juking the Stats . medium.com. January 20, 2016 . August 14, 2024.