Comprehensive Analysis of the Loss Landscape of Deep Linear Neural Networks

The loss landscape of deep linear neural networks has no spurious local minima, but contains a diverse set of strict and non-strict saddle points that can play a role in the dynamics of first-order optimization algorithms.