מושגי ליבה
The authors propose a new projection-free decentralized optimization method, the Inexact Primal-Dual Sliding (I-PDS) algorithm, that achieves communication efficiency and reduces the number of data oracle calls compared to prior work.
תקציר
The authors address the problem of decentralized optimization, where workers in a network collaborate to minimize a sum of local objective functions while preserving the privacy of local data. They propose the I-PDS algorithm, which leverages the conditional gradient sliding method to solve the linear optimization subproblems more efficiently than prior projection-based methods.
Key highlights:
- The I-PDS algorithm achieves gradient sampling complexity that is independent of the graph topology, unlike prior consensus-based methods.
- The algorithm can handle stochastic gradients, making it more robust to noise and suitable for machine learning applications.
- The authors provide theoretical analysis showing that I-PDS achieves optimal gradient complexity and linear oracle complexity for both convex and strongly convex settings.
- Numerical experiments on logistic regression demonstrate the advantages of I-PDS in terms of reduced data oracle access compared to prior methods.
The authors also discuss the effects of different graph topologies on the performance of I-PDS and the prior consensus-based method, showing that I-PDS is more robust to the graph structure.
סטטיסטיקה
The number of gradient samples required after 100 outer iterations for the Stochastic I-PDS method is 21200, which is significantly smaller than the 2 × 10^7 required for the DeFW and Deterministic I-PDS methods.
ציטוטים
"Our proposed method leverages an inexact primal-dual sliding framework (I-PDS), which is inspired by [46] for convex decentralized optimization. Different from [46], which assumes each constrained subproblem can be solved exactly, our I-PDS framework only requires the constrained subproblem to be solved approximately, which can be done by applying the conditional gradient sliding method in [8]."
"Compared to the prior work [1], our method leads to a significant reduction in terms of data oracle calls."