Stochastic Dynamics Reveal Conservative Sharpening and a Stochastic Edge of Stability in Neural Network Training

High-dimensional analysis reveals that stochastic gradient descent (SGD) leads to a conservative sharpening of the loss Hessian eigenvalues, and the existence of a stochastic edge of stability (S-EOS) that is distinct from the deterministic edge of stability observed in full-batch training.