Exploring Low-Dimensional Manifold in Deep Network Training

Training trajectories of deep networks explore an effectively low-dimensional manifold, revealing insights into the optimization process.