Adaptive Consensus Step-size for Decentralized Deep Learning with Communication Compression

AdaGossip, a novel decentralized learning method, dynamically adjusts the consensus step-size based on the compressed model differences between neighboring agents to improve performance under constrained communication.