Convergence and Discretization of Stochastic Gradient-Momentum Processes for Machine Learning Optimization
This paper proposes and analyzes a continuous-time model for stochastic gradient descent with momentum, exploring its convergence properties and proposing a stable discretization scheme for practical application in machine learning optimization.
Robust Stochastic Optimization Using a Regularized Progressive Hedging Algorithm for Energy Management Systems
This paper introduces a novel Regularized Progressive Hedging Algorithm (RPHA) for robust stochastic optimal control, enhancing out-of-sample robustness in energy management systems by incorporating variance penalization and demonstrating superior performance compared to standard methods like Model Predictive Control (MPC) and standard PHA.
On the ReLU Lagrangian Cuts for Stochastic Mixed Integer Programming: A Tight Approximation Approach
This paper introduces ReLU Lagrangian cuts, a new family of nonlinear cuts that provide a tight approximation for solving two-stage stochastic mixed-integer programs (SMIPs) with general mixed-integer decisions in the first stage.
Almost Sure Convergence of Stochastic Hamiltonian Descent Methods: Analysis Under Various Smoothness and Noise Assumptions
This research paper proves the almost sure convergence of a class of stochastic optimization algorithms, inspired by Hamiltonian dynamics, to stationary points of an objective function under various smoothness and noise conditions, including L-smoothness, (L0, L1)-smoothness, and heavy-tailed noise.
單迴圈隨機演算法解決最大結構化弱凸函數差異問題
本文提出了一種名為 SMAG 的單迴圈隨機演算法，用於解決一類非光滑、非凸的 DMax 優化問題，並在理論上證明了其具有與當前最佳演算法相同的非漸進收斂速度。
Single-Loop Stochastic Algorithms for Optimizing the Difference of Weakly Convex Functions with a Max-Structure (SMAG)
This paper introduces SMAG, a novel single-loop stochastic algorithm designed to efficiently solve a class of non-smooth, non-convex optimization problems, specifically focusing on the difference of weakly convex functions with a max-structure (DMax).
Improving the Stochastic Cubic Newton Method Using Momentum for Non-Convex Optimization
Incorporating a specific type of momentum into the Stochastic Cubic Newton method significantly improves its convergence rate for non-convex optimization problems, enabling convergence for any batch size, including single-sample batches.
Sign-Based Stochastic Variance Reduction for Non-Convex Optimization with Improved Convergence Rates
This paper introduces novel sign-based stochastic variance reduction algorithms for non-convex optimization, achieving improved convergence rates compared to existing sign-based methods, both in centralized and distributed settings.
Error Estimates Between Stochastic Gradient Descent with Momentum and Underdamped Langevin Diffusion: A Quantitative Analysis
This research paper establishes a quantitative error estimate between Stochastic Gradient Descent with Momentum (SGDm) and Underdamped Langevin Diffusion in terms of 1-Wasserstein and total variation distances, demonstrating the close relationship between these two optimization methods.
Consensus-Based Particle Swarm Optimization for Stochastic Optimization Problems: A Mean-Field Analysis
This paper proposes and analyzes two novel consensus-based particle swarm optimization algorithms for solving stochastic optimization problems, leveraging mean-field approximations to establish their theoretical foundations and convergence properties.
© 2024 by Linnk AI