The paper studies the sample average approximation (SAA) method for solving convex or strongly convex stochastic programming (SP) problems. Under common regularity conditions, the authors show that SAA's sample complexity can be completely free from any quantification of metric entropy, leading to a significantly more efficient rate with dimensionality compared to most existing results.
The key findings are:
SAA's sample complexity matches exactly with that of the canonical stochastic mirror descent (SMD) method under comparable assumptions, rectifying a persistent theoretical discrepancy between the two mainstream solution approaches to SP.
The authors provide the first, large deviations-type sample complexity bounds for SAA that are completely free from any metric entropy terms in the light-tailed settings. These bounds exhibit a significantly better growth rate with problem dimensionality compared to the state-of-the-art.
The authors identify cases where SAA's theoretical efficacy may even outperform SMD in non-Lipschitzian scenarios where neither the objective function nor its gradient admits a known upper bound on the Lipschitz constant.
Overall, the results demonstrate a new level of SAA's innate, SMD-comparable dimension-insensitivity that has not been uncovered thus far in the literature.
To Another Language
from source content
arxiv.org
Key Insights Distilled From
by Hongcheng Li... at arxiv.org 09-26-2024
https://arxiv.org/pdf/2401.00664.pdfDeeper Inquiries