Adversarial Sparse Teacher (AST) is a new approach to safeguard teacher models against knowledge theft through Knowledge Distillation. By incorporating sparse outputs of adversarial examples, AST aims to mislead adversaries attempting to extract information from the teacher model. The method focuses on reducing the relative entropy between original and adversarially perturbed outputs while maintaining high accuracy. AST leverages a unique loss function, Exponential Predictive Divergence (EPD), to enhance model robustness against stealing attacks. Experimental results demonstrate the effectiveness of AST in complex architectures and datasets, outperforming other strategies in fully disclosed model scenarios.
AST's responses are deliberately misleading, consistently providing incorrect information to deter adversaries. The method significantly impairs adversaries' performance when they have complete knowledge, including access to training data. AST is particularly effective in scenarios where adversaries have full access, showcasing its superiority over other strategies.
The study also introduces EPD loss function utilized in AST training, proving effective in empirical results. Future research is needed to refine this approach and explore its broader implications for computational efficiency and adaptability across various architectures.
翻譯成其他語言
從原文內容
arxiv.org
深入探究