Frequency Attention for Knowledge Distillation: Enhancing Student Learning with Frequency Domain

Frequency attention in the frequency domain enhances knowledge transfer in distillation by encouraging students to mimic teacher features.