מושגי ליבה
The residual loss in Physics-Informed Neural Networks (PINNs) can be globally minimized by using wide neural networks with activation functions that have well-behaved high-order derivatives.
תקציר
The authors analyze the residual loss in PINNs, which is inherently different from common supervised learning tasks due to the application of a differential operator. They study the characteristics of the residual loss at critical points to find conditions that enable effective training of PINNs.
The key findings are:
Under certain conditions, the residual loss of PINNs can be globally minimized by a wide neural network with a width equal to or greater than the number of collocation training points.
The residual loss for a k-th order differential operator is optimally minimized when using an activation function with a bijective k-th order derivative. This provides a guideline for selecting effective activation functions, justifying the use of sinusoidal activations.
The authors verify their theoretical findings through extensive experiments on several PDEs, including the Transport, Wave, Helmholtz, and Klein-Gordon equations. They show that wide PINNs with sinusoidal activations significantly outperform narrow networks and those with common activation functions like Tanh.
סטטיסטיקה
The width of the neural network should be equal to or greater than the number of collocation training points for the residual loss to be globally minimized.
ציטוטים
"Under certain conditions, the residual loss of PINNs can be globally minimized by a wide neural network with a width equal to or greater than the number of collocation training points."
"The residual loss for a k-th order differential operator is optimally minimized when using an activation function with a bijective k-th order derivative."