
Catapults in SGD führen zu besserer Generalisierung durch Feature-Learning.


coremsg

catapults-in-sgd-auswirkungen-auf-generalisierung-durch-feature-learning


Catapults in SGD: Auswirkungen auf Generalisierung durch Feature-Learning


title_rewrite


The author explains how spikes in training loss during SGD are caused by catapult dynamics in the top eigenspace of the tangent kernel. Furthermore, they demonstrate that these catapults lead to better generalization through increased alignment with the Average Gradient Outer Product (AGOP).


Understanding-Catapult-Dynamics-in-Stochastic-Gradient-Descent


Understanding Catapult Dynamics in Stochastic Gradient Descent
