Editing 2899: Goodhart's Law
Warning: You are not logged in. Your IP address will be publicly visible if you make any edits. If you log in or create an account, your edits will be attributed to your username, along with other benefits.
The edit can be undone.
Please check the comparison below to verify that this is what you want to do, and then save the changes below to finish undoing the edit.
Latest revision | Your text | ||
Line 10: | Line 10: | ||
==Explanation== | ==Explanation== | ||
+ | {{incomplete|Created by a METRIC OF METRICS PER METRIC METRIC (IN <s>IMPERIAL</s> US CUSTOMARY UNITS) - Please change this comment when editing this page. Do NOT delete this tag too soon.}} | ||
− | In this comic, [[White Hat]] suggests creating a | + | In this comic, [[White Hat]] recursively suggests creating a metric, "number-of-metrics-that-have-become-targets," and making it a target. |
− | First, | + | First, the comic introduces and defines {{w|Goodhart's Law}}, which is the observation that when a metric — a {{w|performance indicator|measure of performance}} — becomes a goal, efforts will be unhelpfully directed to improving that ''metric'' at the expense of systemic objectives. |
For example, imagine a scenario in which a car dealership is looking to grow profits, and its managers decide to focus on increasing a component metric of profit: how many cars it sells. So they offer a bonus to their salespeople to sell more cars. But then the salespeople offer deep discounts to rack up sales, rendering the car sales unprofitable. This example shows how a ''metric'' (cars sold) can become the ''target'', replacing the real target, profit growth, if individual incentives are not properly managed. | For example, imagine a scenario in which a car dealership is looking to grow profits, and its managers decide to focus on increasing a component metric of profit: how many cars it sells. So they offer a bonus to their salespeople to sell more cars. But then the salespeople offer deep discounts to rack up sales, rendering the car sales unprofitable. This example shows how a ''metric'' (cars sold) can become the ''target'', replacing the real target, profit growth, if individual incentives are not properly managed. | ||
− | + | White Hat's suggestion could be a good or a bad idea. It all depends on how the incentive is implemented: | |
− | + | * A '''good implementation''' would award bonuses only for finding metrics which truly aren't serving their purpose, so the organization's managers could fix the measurement issues. If bonuses are awarded only for approved submissions and the identifications result in real improvements, the organization will benefit. | |
− | * A ''' | + | * A '''bad implementation''' would offer a bonus to every identification — regardless of quality. This would incentivise the removal of metrics that do still serve a useful purpose, despite the development of target behaviour around them (reality is not quite as black and white as Goodhart's Law suggests), and perhaps even the ''creation'' of new metrics-as-targets for the sole purpose of then removing them and collecting the bounty. |
− | + | The title text imagines this '''bad implemention''', leading to the creation of a new metric (metric changes per hour) and the organization identifying — and ''replacing'' — hundreds of metrics per hour, crowding out actual focus on the organization's true goals. It's the ultimate example of "change for change's sake." | |
− | + | Part of the joke is that White Hat's original suggestion -- the one causing the issue and that SHOULD be replaced -- seems to be ironically surviving the replacement of hundreds of other metrics. | |
− | |||
− | Part of the joke is that White Hat's original suggestion | ||
This comic illustrates that the thoughtless combination of Goodhart's Law and poorly designed incentives can have ruinous results for an organization. | This comic illustrates that the thoughtless combination of Goodhart's Law and poorly designed incentives can have ruinous results for an organization. | ||
The proper usage of organizational metrics and incentives is the focus of {{w|managerial accounting}}, a field within organizational management. | The proper usage of organizational metrics and incentives is the focus of {{w|managerial accounting}}, a field within organizational management. | ||
− | |||
− | |||
− | |||
− | |||
− | |||
===Additional examples of Goodhart's Law=== | ===Additional examples of Goodhart's Law=== | ||
− | * | + | * A school's exam results may ''suggest'' how well the school works with its pupils, but may lead to rigidly "teaching to the exams" and lead to less enjoyment and ability of life-long learning, or even flexibility in non-academic activities. |
− | + | * A hospital measures inpatient ''Length of Stay'' because shorter stays save money and also free up beds for any admitted patients waiting in the ER. But if improperly incentivized, doctors may discharge inpatients too early, causing some to "bounce back" to the hospital as a costly readmission. | |
− | * A hospital measures inpatient ''Length of Stay'' because shorter stays save money and free up beds for | + | * A call center measures the number of calls handled per hour, but poorly decides to overly incentivize this metric to make the workers more productive; that leads workers to cut calls short, frustrating customers. |
− | * A call center measures the number of calls handled per hour | ||
− | |||
==Transcript== | ==Transcript== |