Learning Dynamics of Two-Layer Neural Networks: Incremental Approximation of Target Functions

The learning dynamics of two-layer neural networks exhibits a separation of timescales and incremental approximation of the target function, where the network learns simpler polynomial approximations before converging to the full target function.