Agnostic analysis of overfitting costs in kernel ridge regression reveals insights into benign, tempered, and catastrophic overfitting scenarios.