Войти
аналитика - Feature Learning in Neural Networks
The Role of Gradient Descent in Feature Learning: Aligning Weights with Pre-Activation Tangent Features in Neural Networks
The Neural Feature Ansatz (NFA), which observes a correlation between the Gram matrix of weights (NFM) and the average gradient outer product (AGOP) in trained neural networks, emerges from the alignment of weight matrices with pre-activation tangent kernel features driven by gradient descent.
Feature Learning in Two-Layer Neural Networks: A Local Analysis of Gradient Descent with Regularization
This research paper demonstrates that gradient descent, when applied to two-layer neural networks with a carefully regularized objective, can learn useful features not only in the early stages of training but also in the later stages, leading to convergence to the ground-truth features.
Продукты
© 2024 by Linnk AI