Convergence Analysis of RMSProp and Adam Optimizers for Generalized-smooth Non-convex Optimization with Affine Noise Variance

This paper provides the first tight convergence analyses for RMSProp and Adam optimizers in non-convex optimization under the most relaxed assumptions of coordinate-wise generalized smoothness and affine noise variance.