Efficient Federated Learning with Communication Compression: Stochastic Controlled Averaging Algorithms for Arbitrary Data Heterogeneity and Partial Participation
The core message of this work is to develop new compressed federated learning approaches, SCALLION and SCAFCOM, that are practical to implement, robust to arbitrary data heterogeneity and partial client participation, support both biased and unbiased compressors, and exhibit superior theoretical convergence guarantees compared to prior methods.
Accelerating Federated Learning Convergence through Synthetic Data Shuffling under Data Heterogeneity
Shuffling a small fraction of synthetic data across clients can quadratically reduce the gradient dissimilarity and lead to a super-linear speedup in the convergence of federated learning algorithms under data heterogeneity.
Privacy-Preserving Federated Primal-Dual Learning for Non-convex and Non-smooth Problems with Bidirectional Model Sparsification
The authors propose two novel privacy-preserving federated primal-dual learning algorithms, DP-FedPDM and BSDP-FedPDM, to efficiently solve non-convex and non-smooth federated learning problems while considering communication efficiency and privacy protection.
Dynamic Regularized Sharpness-Aware Minimization for Consistent and Smooth Federated Learning
FedSMOO jointly optimizes for global consistency and a smooth loss landscape to efficiently improve performance in federated learning, especially on heterogeneous datasets.
© 2024 by Linnk AI