The article discusses the behavior of stochastic approximation algorithms, proving exponential concentration bounds when progress is proportional to the step size. It contrasts asymptotic normality with exponential concentration results and extends results on Markov chains to stochastic approximation. The analysis applies to various algorithms like Projected Stochastic Gradient Descent, Kiefer-Wolfowitz, and Stochastic Frank-Wolfe. The content delves into sharp convex functions, geometric ergodicity proofs, and linear convergence rates. Exponential distribution bounds are explored in contrast to Gaussian limits typically seen in stochastic optimization literature.
The structure of the content is as follows:
Till ett annat språk
från källinnehåll
arxiv.org
Djupare frågor