Training Data Influence: Quantifying the Impact of Individual Instances on Model Predictions

Core Concepts

The core message of this article is to provide a comprehensive survey of training data influence analysis methods, which quantify the impact of individual training instances on model predictions. These methods help demystify the relationship between training data and model behavior in complex, overparameterized models.

Abstract

This paper provides a comprehensive survey of training data influence analysis methods. It begins by formalizing the various definitions of training data influence, including pointwise influence, group influence, and expected influence. The paper then organizes the state-of-the-art influence analysis methods into a taxonomy and describes each method in detail, comparing their underlying assumptions, strengths, weaknesses, and complexities.
The key highlights and insights from the paper are:

Influence analysis partially demystifies the relationship between training data and model predictions in complex, overparameterized models by quantifying the effect of each training instance on the final model.

Measuring influence exactly can be computationally intractable, so influence estimation methods that approximate the true influence are commonly used in practice. These estimators make different trade-offs and assumptions.

The paper categorizes influence analysis methods into two broad classes: retraining-based methods that measure influence by repeatedly retraining the model, and gradient-based estimators that estimate influence using the alignment of training and test instance gradients.

Retraining-based methods like leave-one-out (LOO) influence are simple and model-agnostic but have high computational costs. Downsampling and Shapley value methods aim to reduce this cost while maintaining accuracy.

Gradient-based estimators like influence functions and TracIn make stronger assumptions about the model and training process but can be more efficient. They estimate influence without full retraining.

Influence analysis has been applied to a variety of tasks, including data cleaning, model interpretability, and adversarial robustness. The paper discusses future research directions to make influence analysis more useful in practice.

Stats

"Training data anomalies may have a natural cause such as distribution shift, measurement error, or non-representative samples drawn from the tail of the data distribution."
"Today's large datasets also generally overrepresent established and dominant viewpoints, encoding and exhibiting biases based on protected characteristics like gender, race, and disability."
"For overparameterized deep models, the causal relationship between training data and model predictions is increasingly opaque and poorly understood."

Quotes

"If all we have is a 'black box' it is impossible to understand causes of failure and improve system safety."
"Without good training data, nothing else works."
"Near-zero training loss occurs because deep models often memorize some training instances."

Key Insights Distilled From

Training Data Influence Analysis and Estimation

by Zayd Hammoud... at arxiv.org 04-02-2024

https://arxiv.org/pdf/2212.04612.pdf

Deeper Inquiries

How can influence analysis methods be extended to handle the stochasticity inherent in modern deep learning training

Incorporating the stochasticity present in modern deep learning training into influence analysis methods is crucial for obtaining more accurate and reliable results. One approach to extend these methods is to incorporate ensemble techniques. By training multiple models with different random initializations or data shuffling orders, we can capture the variability in model predictions due to stochasticity. This ensemble of models can then be used to estimate the influence of training instances more robustly.
Another way to handle stochasticity is to introduce randomness into the influence estimation process itself. This can be achieved by sampling multiple subsets of the training data during the influence analysis and averaging the results. By considering the variability in influence estimates across different samples, we can obtain a more comprehensive understanding of the impact of training instances on the model.
Furthermore, techniques such as bootstrapping can be employed to create multiple bootstrap samples from the training data and estimate the influence on each sample. By aggregating the results from these bootstrap samples, we can account for the uncertainty introduced by the stochastic nature of deep learning training.

What are the potential downsides or unintended consequences of using influence analysis to identify and remove "influential" training instances

While influence analysis can provide valuable insights into the impact of individual training instances on model performance, there are potential downsides and unintended consequences associated with using this approach to identify and remove influential training instances.
One major concern is the risk of overfitting the model to the training data. By selectively removing influential instances based on their impact on the model, there is a possibility of biasing the model towards the remaining data points. This can lead to a loss of generalization performance and an increase in model sensitivity to outliers in the test data.
Another downside is the potential for introducing unintended biases into the model. If influential instances are removed based on certain criteria, such as their effect on model predictions, there is a risk of inadvertently reinforcing existing biases in the data. This can result in a model that is less fair and more prone to making discriminatory decisions.
Additionally, the process of identifying and removing influential training instances can be computationally expensive and time-consuming, especially for large datasets and complex models. This can limit the scalability and practicality of using influence analysis for data curation purposes.

How can influence analysis be combined with other techniques like active learning or dataset curation to improve the overall quality and representativeness of training data

Combining influence analysis with other techniques like active learning and dataset curation can enhance the overall quality and representativeness of training data in machine learning models.
Active learning can be used in conjunction with influence analysis to iteratively select the most informative training instances for labeling. By considering both the influence of training instances on the model and their informativeness for improving model performance, active learning can help prioritize the annotation of data points that are most beneficial for model training.
Dataset curation techniques, such as coreset selection and data pruning, can be integrated with influence analysis to identify and retain the most relevant and diverse training instances. By leveraging the insights from influence analysis to identify influential instances and combining them with dataset curation methods, we can create a more balanced and representative training dataset.
Furthermore, incorporating domain knowledge and expert input into the data curation process can help ensure that the selected training instances align with the objectives and requirements of the specific machine learning task. This hybrid approach can lead to improved model performance, robustness, and fairness by leveraging the strengths of each technique in a complementary manner.