Assessing the Probabilistic Fit of Neural Networks in Regression Tasks Using Conditional Congruence

Calibration is an insufficient metric for evaluating the probabilistic fit of neural networks in regression tasks, and this paper proposes conditional congruence, measured by Conditional Congruence Error (CCE), as a more robust alternative for assessing point-wise uncertainty and identifying model shortcomings.