Federated Distillation: Enhancing Collaborative Learning by Transferring Knowledge Across Heterogeneous Devices
Federated Distillation (FD) integrates knowledge distillation into federated learning to enable more flexible knowledge transfer between clients and the server, surpassing the mere sharing of model parameters. FD mitigates the communication costs associated with training large-scale models and eliminates the need for identical model architectures across clients and the server.