Centrala begrepp
A novel single-loop algorithm, SLDBO, is proposed for efficiently solving decentralized bilevel optimization problems without any heterogeneity assumptions, and it achieves the best-known convergence rate.
Sammanfattning
The paper proposes a novel single-loop algorithm, called SLDBO, for efficiently solving decentralized bilevel optimization (DBO) problems. The key features of SLDBO are:
- It has a single-loop structure, unlike existing DBO algorithms that require a double-loop structure.
- It only needs two matrix-vector multiplications per iteration, which is computationally efficient.
- It does not make any assumptions related to data heterogeneity, in contrast to existing DBO and federated bilevel optimization algorithms.
The convergence rate analysis of SLDBO shows that it achieves the best-known sublinear convergence rate of O(1/K) for a stationarity measure, without requiring any heterogeneity assumptions.
The paper also presents experimental results on hyperparameter optimization problems using both synthetic and MNIST datasets. The results demonstrate the efficiency and effectiveness of the proposed SLDBO algorithm, especially in high-dimensional and heterogeneous data settings.
Statistik
The paper does not provide any specific numerical data or statistics. The key results are the theoretical convergence rate guarantees and the experimental comparisons between SLDBO and other DBO algorithms.