Quantifying Information Leakage in Machine Learning Models via Leave-One-Out Distinguishability

We introduce an analytical framework to quantify the changes in a machine learning algorithm's output distribution following the inclusion or removal of a few data points in its training set, a notion we define as leave-one-out distinguishability (LOOD). This allows us to measure data memorization, information leakage, and the influence of training data points in machine learning.