Detect metric manipulation and gaming
Metrics inevitably shape behavior, sometimes in unexpected ways. When teams face pressure to improve specific numbers, they may optimize for the metric rather than the underlying business goal. This is a phenomenon known as Goodhart's Law: "When a measure becomes a target, it ceases to be a good measure."[1]
Metric manipulation takes various forms, from harmless optimization to harmful distortion. For example, a team might focus on increasing session duration might add unnecessary friction to tasks, making the product less efficient but "improving" the metric. Similarly, focusing exclusively on short-term conversion rates or choosing revenue as your North Star metric could lead to aggressive tactics that boost numbers while damaging long-term customer relationships.
Detecting metric gaming requires establishing balancing metrics that catch unintended consequences. If optimizing for speed, also measure accuracy. If tracking conversion, monitor retention as well. Creating these health metric pairs helps ensure improvements in one area don't come at the expense of others. Additionally, regularly reviewing user feedback provides qualitative signals that can reveal when metric improvements don't align with actual user experience. Also, choose metrics that resist gaming by measuring outcomes rather than activities. For instance, "number of features shipped" encourages quantity over quality, while "percentage of features with high adoption rates" rewards meaningful innovation.
