Creating Stable and Efficient Adaptive Algorithms for Neural Networks: Introducing Polygonal Unadjusted Langevin Algorithms
The author introduces Polygonal Unadjusted Langevin Algorithms as a solution to the exploding and vanishing gradient problems in deep learning, providing stability and efficiency.
Efficient Methods for Solving Semidefinite Programs
The author introduces a new storage-optimal first-order method for solving semidefinite programs with low-rank solutions, strict complementarity, and known subspace restrictions.
Primal-Dual Alternating Proximal Gradient Algorithms for Nonconvex Minimax Problems with Coupled Linear Constraints
The authors propose novel algorithms for solving nonconvex minimax problems with coupled linear constraints, providing iteration complexity guarantees.
Projected Gradient Descent and Bouligand Stationary Points Analysis
The authors analyze the convergence of Projected Gradient Descent at Bouligand stationary points, proving their properties under specific conditions.
FOSI: Hybrid First and Second Order Optimization Study at ICLR 2024
FOSI is a novel meta-algorithm that enhances first-order optimizers by incorporating second-order information efficiently.
Non-Convex Relaxation and 1/2-Approximation Algorithm for Chance-Constrained Binary Knapsack Problem
The author introduces a non-convex relaxation approach for the chance-constrained binary knapsack problem, providing upper bounds as tight as other continuous relaxations. A polynomial-time algorithm is proposed to solve this relaxation efficiently.
Sharpened Lazy Incremental Quasi-Newton Method Analysis
The author introduces the Sharpened Lazy Incremental Quasi-Newton Method (SLIQN) to address shortcomings in existing incremental methods, achieving explicit superlinear convergence and superior empirical performance at a per-iteration O(d2) cost.
Analyzing Improvements to ADAM Optimizer with IMEX Time-Stepping Approach
Improving the Adam optimizer through IMEX methods enhances neural network training.
First-order Optimization Algorithms: Formal Complexity Analysis and Convergence Rates
Optimization algorithms' convergence rates are formally analyzed using the Lean4 theorem prover, enhancing mathematical representation.
Reinforcement Learning Guided Hybrid Evolutionary Algorithm for Latency Location Routing Problem
Proposing a novel algorithm combining reinforcement learning and evolutionary strategies to solve the latency location routing problem efficiently.
