Understanding the Regimes of Stochastic Gradient Descent

The author explores the impact of hyperparameters on neural network training dynamics, revealing distinct regimes and their dependence on data size.