The paper introduces Self-Guided Label Refinement (SGLR) to improve adversarial robustness by refining label distributions. It identifies the issue of robust overfitting in adversarial training due to noisy labels and proposes a method to combat it. SGLR self-refines accurate labels from over-confident hard labels, incorporating knowledge from previous models without external teachers. Experimental results show improved accuracy and robust performance across datasets, attack types, and architectures. The study also delves into the memorization effect of noisy labels during training and proposes a strategy for label refinement based on information theory. The method is compared against other techniques like label smoothing and knowledge distillation, showing superior performance in reducing generalization gaps and achieving higher robust accuracy under various adversaries.
To Another Language
from source content
arxiv.org
Key Insights Distilled From
by Daiwei Yu,Zh... at arxiv.org 03-15-2024
https://arxiv.org/pdf/2403.09101.pdfDeeper Inquiries