The paper proposes an adaptive federated learning (FL) framework called FedAgg that addresses the challenges of client heterogeneity and slow convergence in traditional FL methods. The key innovations are:
Adaptive Learning Rate: FedAgg introduces an adaptive learning rate for each client, which is determined by considering the aggregated gradients of all clients and the deviation between the local and average model parameters. This helps alleviate the negative impact of client drifting and data heterogeneity.
Mean-Field Estimation: Since clients cannot directly access each other's local information during training, FedAgg introduces two mean-field terms to estimate the average of local gradients and parameters. This allows each client to independently compute its optimal adaptive learning rate without requiring explicit communication.
Theoretical Analysis: The authors provide a rigorous theoretical analysis to prove the existence of the mean-field terms and the convergence of the proposed FedAgg algorithm. They derive a closed-form expression for the adaptive learning rate and establish an upper bound on the convergence rate.
Empirical Evaluation: Extensive experiments on various datasets demonstrate that FedAgg outperforms state-of-the-art FL methods in terms of model performance and convergence speed, under both IID and non-IID data distributions.
The FedAgg framework effectively addresses the challenges of client heterogeneity and slow convergence in federated learning by introducing an adaptive learning rate mechanism and leveraging mean-field theory to estimate global statistics without direct communication. The theoretical analysis and empirical results showcase the superiority of the proposed approach.
To Another Language
from source content
arxiv.org
Principais Insights Extraídos De
by Wenhao Yuan,... às arxiv.org 04-15-2024
https://arxiv.org/pdf/2303.15799.pdfPerguntas Mais Profundas