Implicit Regularization of Gradient Flow on One-Layer Softmax Attention: Analysis and Insights

Implicit regularization through gradient flow minimizes nuclear norm of attention weights.