核心概念
The proposed Client-Supervised Federated Learning (FedCS) framework learns a unified global model that can encode client-specific biases and make personalized predictions without requiring extra fine-tuning or client-specific parameters.
要約
The paper proposes a novel Client-Supervised Federated Learning (FedCS) framework to address the challenge of model personalization in federated learning settings.
Key highlights:
- FedCS aims to learn a single robust global model that can achieve competitive performance to personalized models on unseen/test clients, without requiring extra fine-tuning or client-specific parameters.
- FedCS introduces a Representation Alignment (RA) mechanism to align the latent representation space such that it encodes client-specific biases, while still preserving client-agnostic knowledge.
- A client-supervised optimization framework is designed to optimize the RA module collaboratively under the federated learning setting, without the need to collect privacy-sensitive client-level statistics.
- Experiments on benchmark datasets with label-shift and feature-shift heterogeneity show that FedCS can learn a global model that is more robust to different data distributions compared to other personalized federated learning methods.
The FedCS framework presents a new direction for personalized federated learning, moving away from the traditional approach of learning multiple client-specific models, towards a one-model-for-all strategy that can capture personalization through the learned representation space.
統計
The global model learned with FedCS achieves weighted AUC scores of 99.72 and weighted F1 scores of 93.72 on the MNIST dataset, outperforming other personalized federated learning methods.
On the CIFAR-10 dataset, the FedCS global model achieves weighted AUC scores of 93.72 and weighted F1 scores of 69.48, significantly better than the baselines.
For the feature-shift Digit-5 dataset, the FedCS global model achieves weighted AUC scores of 98.74 and weighted F1 scores of 87.89, demonstrating strong robustness to distribution shifts.
引用
"FedCS can learn a robust FL global model for the changing data distributions of unseen/test clients. The FedCS's global model can be directly deployed to the test clients while achieving comparable performance to other personalized FL methods that require model adaptation."
"The one-model-for-all personalization can form a new topic to advance existing personalized federated learning research. It is foreseeing more discussion and exploration can be conducted in this new one-model-for-all personalized federated setting."