
Proposing NAYER, a novel method for efficient data-free knowledge distillation using noisy layer generation and meaningful label-text embeddings.


coremsg

nayer-noisy-layer-data-generation-for-efficient-and-effective-data-free-knowledge-distillation


NAYER: Noisy Layer Data Generation for Efficient and Effective Data-free Knowledge Distillation


title_rewrite


提案されたNoisy Layer Generation（NAYER）は、ランダムなノイズ入力からのサンプル生成における課題を解決し、高品質なサンプルを効率的に生成する方法を提供します。



The core message of this paper is to introduce a novel causal inference perspective to handle the distribution shifts between the substitution and original data in the data-free knowledge distillation (DFKD) task, and propose a Knowledge Distillation Causal Intervention (KDCI) framework to de-confound the biased student learning process.


de-confounded-data-free-knowledge-distillation-for-handling-distribution-shifts


De-confounded Data-free Knowledge Distillation for Handling Distribution Shifts
