Core Concepts
Ensemble DeepONet architectures, including a novel Partition-of-Unity Mixture-of-Experts (PoU-MoE) trunk, can significantly improve the accuracy of operator learning compared to standard DeepONets, especially for problems involving output functions with steep spatial gradients.
Abstract
The paper presents a novel deep operator network (DeepONet) architecture called the ensemble DeepONet, which allows for enriching the trunk network of a single DeepONet with multiple distinct trunk networks. This trunk enrichment enables greater expressivity and generalization capabilities over a range of operator learning problems.
The authors also introduce a spatial Mixture-of-Experts (MoE) DeepONet trunk network architecture, called the PoU-MoE trunk, that utilizes a Partition-of-Unity (PoU) approximation to promote spatial locality and model sparsity in the operator learning problem.
The authors first prove that both the ensemble and PoU-MoE DeepONets are universal approximators. They then demonstrate that ensemble DeepONets containing a trunk ensemble of a standard trunk, the PoU-MoE trunk, and/or a Proper Orthogonal Decomposition (POD) trunk can achieve 2-4x lower relative ℓ2 errors than standard DeepONets and POD-DeepONets on both standard and challenging new operator learning problems involving partial differential equations (PDEs) in two and three dimensions.
The key highlights are:
- The ensemble DeepONet provides a powerful and general framework for incorporating basis enrichment in scientific machine learning architectures for operator learning.
- The PoU-MoE formulation offers a natural way to incorporate spatial locality and model sparsity into any neural network architecture.
- Ensemble DeepONets with a combination of global (POD) and local (PoU-MoE) basis functions outperform standalone DeepONets and overparametrized DeepONets, especially on problems with output functions exhibiting steep spatial gradients.
Stats
The relative ℓ2 error on the 2D Darcy flow problem was reduced from 0.857% for the vanilla DeepONet to 0.187% for the ensemble Vanilla-POD-PoU DeepONet.
The relative ℓ2 error on the 2D reaction-diffusion problem was reduced from 0.144% for the vanilla DeepONet to 0.0539% for the ensemble POD-PoU DeepONet.
The relative ℓ2 error on the 3D reaction-diffusion problem was reduced from 0.127% for the vanilla DeepONet to 0.0576% for the ensemble POD-PoU DeepONet.
Quotes
"Ensemble DeepONets containing a trunk ensemble of a standard trunk, the PoU-MoE trunk, and/or a Proper Orthogonal Decomposition (POD) trunk can achieve 2-4x lower relative ℓ2 errors than standard DeepONets and POD-DeepONets on both standard and challenging new operator learning problems involving partial differential equations (PDEs) in two and three dimensions."
"The PoU-MoE formulation provides a natural way to incorporate spatial locality and model sparsity into any neural network architecture."