Główne pojęcia
ProxSkip algorithm can achieve linear speedup in terms of the number of nodes and communication probability for stochastic non-convex, convex, and strongly convex optimization problems.
Streszczenie
The paper revisits the decentralized ProxSkip algorithm and provides a new analysis with a novel proof technique for stochastic non-convex, convex, and strongly convex optimization problems.
Key highlights:
Establishes non-asymptotic convergence rates for ProxSkip under stochastic non-convex, convex, and strongly convex settings. The rates demonstrate that ProxSkip can achieve linear speedup with respect to the number of nodes and communication probability.
Proves that the leading communication complexity of ProxSkip is O(pσ^2/nϵ^2) for non-convex and convex settings, and O(pσ^2/nϵ) for the strongly convex setting, where n is the number of nodes, p is the probability of communication, σ^2 is the noise variance, and ϵ is the desired accuracy level.
Shows that for the strongly convex setting, ProxSkip can achieve linear speedup with network-independent stepsizes, overcoming the limitations of prior analyses.
Demonstrates the robustness of ProxSkip against data heterogeneity while enhancing communication efficiency through local updates. The convergence rates of ProxSkip are comparable to existing state-of-the-art decentralized algorithms incorporating local updates.
Statystyki
The paper does not provide specific numerical data, but rather focuses on theoretical analysis and convergence rates.