The Steepest Perturbed Gradient Descent (SPGD) algorithm enhances traditional gradient descent by incorporating periodic, randomized perturbations to escape local minima and efficiently navigate complex optimization landscapes, demonstrating superior performance in various benchmark tests and a 3D component packing problem.
This paper demonstrates that Polyak's heavy ball method achieves an accelerated local rate of convergence for functions satisfying the Polyak-Łojasiewicz inequality, both in continuous and discrete time, challenging the prevailing notion that strong convexity is necessary for such acceleration.
본 논문에서는 Tikhonov 정규화 단조 흐름의 명시적 이산화에서 파생된 새로운 추가 기울기 방법을 소개하며, 이 방법은 일반 매개변수에 의해 제어되는 앵커 항을 사용하여 강력한 수렴성과 빠른 잔차 감소율을 달성합니다.
This paper proposes a novel class of efficient numerical algorithms, based on SCD semismooth* Newton methods, for minimizing Tikhonov functionals with non-smooth and non-convex penalties, commonly used in variational regularization for solving ill-posed problems.
Power law spectral conditions impact optimization convergence rates for algorithms like GD, SD, HB, and CG.
Utilizing Quantum Langevin Dynamics for optimization problems, proving convergence in convex landscapes.
Adam converges faster than SGDM under non-uniform smoothness conditions.
Adapproxは、ランダム低ランク行列近似を用いてアダムの第二モーメントを効率的に近似することで、メモリ使用量を大幅に削減しつつ、精度と収束速度を維持する新しい最適化手法である。
Adapprox introduces a novel approach using randomized low-rank matrix approximation to optimize memory consumption in training large-scale models.
Proposing a novel algorithm combining reinforcement learning and evolutionary strategies to solve the latency location routing problem efficiently.