The paper introduces the "Dog Walking Theory" to analyze the convergence issues in federated learning (FL). It views the FL process as a dog walking scenario, where the server acts as the dog walker and the clients as the dogs. The goal is to ensure the dogs (clients) arrive at the destination (convergence) while allowing them enough exploration (local training).
The key insight is that existing FL methods lack a "leash" that can guide the convergence of the clients. The authors categorize existing FL methods as "passive" methods that do not have an explicit leash task to control the clients. In contrast, the proposed "FedWalk" algorithm is an "active" FL method that introduces a leash task defined on a public dataset to guide the convergence of the clients.
Specifically, FedWalk has two main steps: 1) the server collaborates with clients to optimize the FL task, and 2) the server optimizes the leash task. The leash task serves as a convergence guidance for the clients, and its strength is controlled by a hyper-parameter τ. The authors provide a theoretical analysis on the convergence of FedWalk, showing that the leash task can accelerate convergence when the heterogeneity between the leash and FL tasks is low.
Experiments on CIFAR-10 and CIFAR-100 datasets under both IID and non-IID settings demonstrate the superiority of FedWalk over state-of-the-art FL methods. FedWalk can also boost the performance of existing FL algorithms when combined with them. The ablation studies further confirm the importance of the leash task and the guiding strength controlled by τ.
다른 언어로
소스 콘텐츠 기반
arxiv.org
더 깊은 질문