Frequency Bias in Neural Networks for Solving Partial Differential Equations: Loss Jump and Multi-stage Descent Phenomenon

Neural networks exhibit different frequency preferences when trained with data loss versus model loss for solving partial differential equations, leading to a loss jump phenomenon and multi-stage descent behavior.