Learning Subgrid-Scale Turbulence Models with Differentiable Physics

Core Concepts

The authors leverage differentiable programming techniques to learn highly effective and versatile subgrid-scale (SGS) models for two-dimensional turbulent flow using physics-inspired deep learning architectures.

Abstract

The content discusses the use of differentiable programming techniques to learn subgrid-scale (SGS) turbulence models for large eddy simulations (LES) of two-dimensional turbulent flow. The key highlights are:

The authors develop an end-to-end differentiable solver that combines deep learning models with traditional numerical methods to learn SGS closures. This "differentiable turbulence" approach allows backpropagation of the a-posteriori error to the model parameters.

Several neural network architectures are evaluated, including multi-layer perceptrons (MLPs), convolutional neural networks (CNNs), Fourier neural operators (FNOs), and a hybrid CNN+FNO model. The analysis shows that capturing small-scale non-local features is critical for effective SGS modeling, while large-scale features can improve pointwise accuracy.

The velocity gradient tensor is decomposed into isotropic, deviatoric, and anti-symmetric components, which are then mapped to the SGS stress through the neural network.

The differentiable physics approach is shown to outperform offline, a-priori learning of the SGS stress. The hybrid solver-in-the-loop approach offers a balance between computational efficiency, accuracy, and generalization.

The learned models are evaluated on a variety of flow configurations, including higher and lower Reynolds numbers and different forcing conditions. The results demonstrate the ability of the models to generalize to unseen flow regimes.

Stats

The following sentences contain key metrics or figures:
The velocity gradient tensor on the LES grid can be mapped directly to the SGS stress via decomposition of the inputs and outputs into isotropic, deviatoric, and anti-symmetric components.
We see that the model can generalize to a variety of flow configurations, including higher and lower Reynolds numbers and different forcing conditions.

Quotes

"The concept of "reverse-mode differentiation" in CFD simulations has existed for decades and is analogous to solving the adjoint problem."
"Differentiable simulations have been integrated with ML as "solver-in-the-loop" approaches particularly for correcting and improving the accuracy of coarse-grained or unresolved simulations, which offers a balance between computational cost and physics-embedding."

Key Insights Distilled From

Differentiable Turbulence

by Varun Shanka... at arxiv.org 03-29-2024

https://arxiv.org/pdf/2307.03683.pdf

Deeper Inquiries

How can the differentiable turbulence approach be extended to three-dimensional turbulent flows, which are more representative of real-world applications?

The extension of the differentiable turbulence approach to three-dimensional turbulent flows involves several key considerations. Firstly, the computational complexity increases significantly in three dimensions due to the additional spatial dimension. This necessitates more advanced neural network architectures and training strategies to handle the higher-dimensional data. One approach is to use 3D convolutional neural networks (CNNs) to capture the spatial features in all three dimensions effectively. Additionally, incorporating Fourier Neural Operators (FNOs) in 3D can help capture the global features of the flow, similar to the hybrid CNN+FNO architecture used in the 2D case.
Furthermore, the inclusion of temporal dynamics becomes crucial in 3D turbulent flows, as the evolution of turbulence over time plays a significant role. Recurrent neural networks (RNNs) or spatiotemporal CNNs can be employed to model the temporal aspects of the flow. The use of physics-informed neural networks, where the underlying physical principles are embedded in the network architecture, can also enhance the model's ability to capture the dynamics of 3D turbulence.
To address the challenges of training data-driven models for 3D turbulent flows, a large and diverse dataset encompassing a wide range of flow conditions and Reynolds numbers is essential. Transfer learning techniques, where models trained on simpler flows are fine-tuned for more complex 3D flows, can also be beneficial. Overall, the extension to three-dimensional turbulent flows requires a combination of advanced neural network architectures, incorporation of temporal dynamics, and robust training strategies to effectively model the complexities of real-world turbulent flows.

What are the potential limitations or drawbacks of the tensor decomposition approach used to map the velocity gradient to the SGS stress, and are there alternative formulations that could be explored?

The tensor decomposition approach used to map the velocity gradient to the subgrid-scale (SGS) stress in the differentiable turbulence framework has certain limitations and drawbacks. One limitation is the assumption of linearity in the relationship between the velocity gradient tensor and the SGS stress, which may not hold true for all turbulent flows. This linear assumption can lead to inaccuracies, especially in flows with complex dynamics or non-linear interactions.
Another drawback is the reliance on predefined tensor bases for the decomposition, which may not capture the full complexity of the SGS stress field. The choice of basis tensors can introduce biases and limitations in the model's ability to generalize to diverse flow conditions. Additionally, the tensor decomposition approach may struggle to capture non-local interactions and anisotropic features in the flow, limiting the model's accuracy in certain scenarios.
Alternative formulations that could be explored include the use of more flexible neural network architectures, such as graph neural networks or attention mechanisms, to directly learn the mapping from the velocity gradient tensor to the SGS stress. These approaches can capture non-linear relationships and complex dependencies in the data without relying on predefined tensor bases. Physics-informed neural networks, where the underlying physical principles are explicitly incorporated into the model, can also enhance the accuracy and robustness of the SGS stress prediction.
Overall, while tensor decomposition provides a structured and interpretable way to model the SGS stress, exploring more flexible and adaptive approaches using neural networks can address the limitations of the tensor-based approach and improve the overall performance of data-driven turbulence models.

Given the success of the hybrid CNN+FNO architecture, are there other ways to combine the strengths of local and global modeling approaches to further improve the generalization and accuracy of data-driven turbulence models?

The success of the hybrid CNN+FNO architecture in data-driven turbulence modeling opens up possibilities for further enhancing the generalization and accuracy of models by combining the strengths of local and global modeling approaches. Several other strategies can be explored to leverage the benefits of both local and global modeling:

Graph Neural Networks (GNNs): GNNs are well-suited for capturing complex relationships in graph-structured data, making them ideal for modeling the interactions in turbulent flows. By incorporating graph-based representations of the flow field, GNNs can combine local and global information effectively.

Attention Mechanisms: Attention mechanisms allow models to focus on different parts of the input data with varying degrees of emphasis. By integrating attention mechanisms into CNNs or FNOs, the model can dynamically adjust its focus on local and global features based on the flow dynamics.

Ensemble Learning: Combining multiple models trained with different architectures or hyperparameters can improve the robustness and generalization of the overall model. Ensemble methods can leverage the strengths of diverse models to achieve better performance.

Adaptive Fusion: Implementing adaptive fusion mechanisms that dynamically combine local and global features based on the characteristics of the flow can enhance the model's ability to capture multiscale interactions. Techniques like adaptive weighting or feature fusion can be explored for this purpose.

Hybrid Physics-Informed Models: Integrating physics-based constraints and domain knowledge into the data-driven models can enhance their interpretability and accuracy. Hybrid models that combine physics-informed constraints with data-driven learning can strike a balance between accuracy and physical consistency.

By exploring these approaches and experimenting with novel combinations of local and global modeling techniques, researchers can further advance the capabilities of data-driven turbulence models for improved generalization and accuracy in real-world applications.