Główne pojęcia
Developing the CAS algorithm for online selective conformal prediction to control FCR in real-time predictive tasks.
Streszczenie
The article introduces the CAS algorithm for online selective conformal prediction to control the false coverage-statement rate (FCR) in real-time predictive tasks. It addresses the issue of temporal multiplicity in selected prediction intervals and provides a framework that ensures exact selection-conditional coverage guarantee. The algorithm wraps around any prediction model and online selection rule to output post-selection prediction intervals, achieving distribution-free regimes. Numerical results confirm the effectiveness of CAS in controlling FCR and narrowing prediction intervals.
Abstract:
Study on post-selection predictive inference in an online fashion.
Importance of controlling real-time FCR in online predictive tasks.
Introduction:
Real-time selection before prediction allocation.
Utilization of conformal inference for valid confidence intervals.
Methodology:
Development of CAS algorithm for adaptive selection.
Calibration set construction based on historical data.
Results:
CAS achieves exact selection-conditional coverage guarantee.
Control of real-time FCR below target level without distribution assumptions.
Applications:
Illustrative application in drug discovery using machine learning tools.
Related Work:
Comparison with existing methods like LORD-CI and e-LOND-CI.
Conclusion:
Organization of paper sections and technical proofs provided in Supplementary Material.
Statystyki
"Since training large-scale machine learning models is time-consuming, we consider that one pre-trained model bµ is given in this work."
"We provide tractable constructions for the calibration set for popular online selection rules."
"CAS can achieve an exact selection-conditional coverage guarantee in the finite-sample and distribution-free regimes."
"For decision-driven selection rules, we prove that CAS can exactly control the real-time FCR below the target level without any distributional assumption."
"For the selection with symmetric thresholds, we also provide an upper bound for the real-time FCR value under mild stability condition on the selection threshold."
"To deal with the distribution shift in online data, we adjust the level of prediction intervals whenever the selection happens by leveraging past feedback at each step."