Confidence bands for tuning curves provide a rigorous statistical basis to compare machine learning models while accounting for the cost of hyperparameter tuning.