Almost-Surely Convergent Stochastic Splitting Algorithms for Monotone Inclusion Problems with Applications to Large-Scale Data Analysis
This research paper introduces novel stochastic splitting algorithms designed to efficiently solve large-scale composite monotone inclusion problems, guaranteeing almost-sure convergence of iterates to a solution without requiring prior knowledge of linear operator norms or imposing restrictive regularity assumptions.
Convergence Analysis and Asymptotic Properties of the λ-SAGA Algorithm with Decreasing Step Size for Stochastic Optimization
The λ-SAGA algorithm, a generalization of the SAGA algorithm, exhibits almost sure convergence and demonstrably reduces asymptotic variance compared to traditional SGD, even without assuming strong convexity or Lipschitz gradient conditions.
Tuning-Free Stochastic Optimization: Achieving Optimal Performance Without Hyperparameter Tuning
Tuning-free algorithms can match the performance of optimally-tuned optimization algorithms with only loose hints on problem parameters.
Polygonal Unadjusted Langevin Algorithms: Overcoming Gradient Challenges in Deep Learning
新しいクラスのLangevinベースのアルゴリズムは、深層学習における勾配の課題を克服する。
Understanding the Optimization and Generalization of Stochastic Optimization Algorithms in Deep Learning
This paper explores the relationship between optimization and generalization in deep learning by considering the stochastic nature of optimizers. By analyzing populations of models rather than single models, it sheds light on the performance of various algorithms.
