Core Concepts
Exponential concentration bounds for stochastic approximation algorithms.
Abstract
The article discusses the behavior of stochastic approximation algorithms, proving exponential concentration bounds when progress is proportional to the step size. It contrasts asymptotic normality with exponential concentration results and extends results on Markov chains to stochastic approximation. The analysis applies to various algorithms like Projected Stochastic Gradient Descent, Kiefer-Wolfowitz, and Stochastic Frank-Wolfe. The content delves into sharp convex functions, geometric ergodicity proofs, and linear convergence rates. Exponential distribution bounds are explored in contrast to Gaussian limits typically seen in stochastic optimization literature.
The structure of the content is as follows:
Introduction to Stochastic Approximation Algorithms.
Analysis of Stochastic Gradient Descent.
Asymptotic Normality vs. Exponential Bounds.
Construction of Exponential Concentration Bounds.
Application to Various Algorithms: Kiefer-Wolfowitz and Frank-Wolfe.
Linear Convergence under Exponential Concentration.
Proofs and Technical Lemmas.
Stats
For any t and ˆt with t ≥ˆt ≥T0 and for any η > 0 such that αˆtη ≤λ then E[eηLt+1|Fˆt] ≤E[eηLT1 |Fˆt] * ∏(k=ˆt)^(t-1) (ρk + D)
E[eηLt+1|Fˆt] ≤E[eηLT1 |Fˆt] * ∏(k=ˆt)^(t-1) (ρk + D)
Quotes
"An OU process is known to have a normal distribution as its limiting stationary distribution."
"The resulting distributions all exhibit exponential tail bounds."