Adaptive Consensus Step-size for Decentralized Deep Learning with Communication Compression
AdaGossip, a novel decentralized learning method, dynamically adjusts the consensus step-size based on the compressed model differences between neighboring agents to improve performance under constrained communication.
Preserving Optimal Variance in Decentralized Neural Network Training to Improve Convergence Efficiency
Averaging uncorrelated neural network models in gossip learning systems can lead to a "vanishing variance" problem, causing significant convergence delays. A variance-corrected model averaging algorithm is proposed to eliminate this issue, enabling gossip learning to achieve convergence efficiency comparable to federated learning.
Decentralized Collaborative Learning Framework for Anomaly Detection with Privacy Leakage Analysis
This paper presents a decentralized collaborative learning framework that incorporates deep variational autoencoders (VAEs) for enhanced anomaly detection, while providing a theoretical analysis on external data privacy leakage when the trained models are shared.
Model Agnostic Peer-to-Peer Learning for Heterogeneous Personalized Models
MAPL jointly learns heterogeneous personalized models and a collaboration graph in a decentralized peer-to-peer setting, without relying on a central server.