Analyzing the Dynamics of Weight Updates in Feed-Forward Neural Networks for Improved Training Outcomes

The weight dynamics of feed-forward neural networks during the training process can be characterized by indicators of local stability, such as Lyapunov exponents and covariant Lyapunov vectors, which can be used to predict the final training loss.