Federated learning optimization

insight - Federated learning optimization

Efficient Federated Learning with Communication Compression: Stochastic Controlled Averaging Algorithms for Arbitrary Data Heterogeneity and Partial Participation

The core message of this work is to develop new compressed federated learning approaches, SCALLION and SCAFCOM, that are practical to implement, robust to arbitrary data heterogeneity and partial client participation, support both biased and unbiased compressors, and exhibit superior theoretical convergence guarantees compared to prior methods.

Accelerating Federated Learning Convergence through Synthetic Data Shuffling under Data Heterogeneity

Shuffling a small fraction of synthetic data across clients can quadratically reduce the gradient dissimilarity and lead to a super-linear speedup in the convergence of federated learning algorithms under data heterogeneity.

Privacy-Preserving Federated Primal-Dual Learning for Non-convex and Non-smooth Problems with Bidirectional Model Sparsification

The authors propose two novel privacy-preserving federated primal-dual learning algorithms, DP-FedPDM and BSDP-FedPDM, to efficiently solve non-convex and non-smooth federated learning problems while considering communication efficiency and privacy protection.

Dynamic Regularized Sharpness-Aware Minimization for Consistent and Smooth Federated Learning

FedSMOO jointly optimizes for global consistency and a smooth loss landscape to efficiently improve performance in federated learning, especially on heterogeneous datasets.

About

Products | Resources

Insights