核心概念
A self-guided framework that automatically detects and mitigates a classifier's reliance on spurious correlations in the data without requiring any prior annotations.
摘要
The paper proposes a novel self-guided framework, called Learning beyond Classes (LBC), to train robust deep neural classifiers without requiring any annotations of spurious correlations in the data.
The key components of the framework are:
-
Automatic Spurious Correlation Detection:
- Leverages a pre-trained vision-language model to automatically detect attributes in the images.
- Proposes a spuriousness score to quantify the likelihood of a class-attribute correlation being spurious and exploited by the classifier.
-
Spuriousness-Guided Training Data Relabeling:
- Constructs a spuriousness embedding space to characterize the classifier's prediction behaviors based on the detected attributes and their spuriousness scores.
- Clusters the training samples in the spuriousness embedding space and relabels them with fine-grained labels to diversify the classifier's outputs.
-
Learning beyond Classes:
- Modifies the classifier architecture to predict the fine-grained labels instead of just class labels.
- Adopts within-class and cross-class balanced sampling strategies to address the imbalanced distribution of different prediction behaviors.
The framework iteratively identifies and mitigates the classifier's reliance on spurious correlations, leading to improved robustness without any prior knowledge of the spurious attributes. Experiments on several real-world datasets demonstrate the effectiveness of the proposed method in outperforming state-of-the-art approaches.
統計資料
The classifier trained with empirical risk minimization (ERM) can achieve high accuracy by exploiting spurious correlations between non-essential attributes and target classes.
Obtaining annotations of spurious correlations typically requires expert knowledge and human supervision, which is a significant barrier in practice.
引述
"Deep neural classifiers tend to rely on spurious cor-relations between spurious attributes of inputs and targets to make predictions, which could jeopar-dize their generalization capability."
"Mitigating the reliance on spurious correlations is crucial for obtaining robust models."