Reliance on metrics is a fundamental challenge for AI
Optimizing metrics is a central aspect of most current artificial intelligence (AI)
approaches, yet overemphasizing metrics leads to manipulation, short-termism, and
other negative consequences. This poses a fundamental challenge within AI. We review
how metrics can go wrong in practice, and we put forward an evidence-based framework
toward mitigating these harms.