The paper introduces a method for evaluating AI/ML models based on multiple criteria, including scientific principles and practical outcomes. It addresses the limitations of ML models compared to scientifically informed theories, emphasizing the importance of generalizability and explainability. The authors propose a multi-criteria evaluation method that allows for holistic model assessment across various criteria. By quantifying desirable characteristics like generalizability, explainability, and adverse impact, this approach aims to incentivize better models and improve model evaluations in different fields. The method originated from critiques of decision-making competitions in Psychology and Cognitive Science, highlighting the need for diverse model types with identifiable process assumptions. Through ordinal ranking and voting rules from computational social choice, this method enables direct comparisons between models based on multiple criteria simultaneously.
To Another Language
from source content
arxiv.org
Key Insights Distilled From
by Jason L. Har... at arxiv.org 03-19-2024
https://arxiv.org/pdf/2403.11840.pdfDeeper Inquiries