Deep Learning

Codes available:

1. Parametric Study
   • Multiparametric Tool
2. Spatial Resolution Enhancement
   • Superresolution Tool
3. Temporal Forecasting
   • Hybrid Generative Model
     • Standard
     • Advanced
   • Purely Deep Learning Models
     • Residual Autoencoder (point forecasting)
     • Variational Autoencoder (probabilistic forecasting)
   • Hybrid Predictive Model: POD-DL
     • POD-DL: Fixed temporal horizon
     • POD-DL: Autoregressive
   • Hybrid Predictive Model: HOSVD-DL
     • HOSVD-DL: Autoregressive
   • Remote Sensing and DLinear
4. Control

Parametric Study

Multiparametric Tool

Numerical simulations of complex fluid phenomena are computationally expensive in terms of computational resources and processing time. Several numerical simulations must be performed to analyze the behavior of the flow under different flow regimes or parameters, and this entire process can take several weeks or even months.

To overcome these limitations, we have developed an innovative tool capable of efficiently generating new fluid dynamics databases for specific flow conditions (parameters) and predicting their behavior over time. This is achieved using a hybrid, fully data-driven Reduced-Order Model (ROM), which combines Higher-Order Modal Decomposition (HOSVD) with Recurrent Neural Networks (RNN) and the Kriging interpolation method, also known as Gaussian Process Regression (GPR).

This framework has been developed using Python’s ML libraries, Tensorflow/Keras, and the PyKrige interpolation library. The tool has been tested on databases obtained through numerical simulations at different flow parameters, such as Reynolds number and Angle of Attack.

Code in progress. Coming soon…

Spatial Resolution Enhancement

Superresolution Tool

Fluid mechanics plays a significant role in most natural and anthropogenic processes. During the last decades, numerical methods such as the Finite Volume Method (FVM) have been widely used to predict the behavior of fluid dynamics phenomena. However, performing accurate numerical simulations for complex flows (such as those in a turbulent regime, multiphase flows, multiscale phenomena, etc.) demands significant computational resources in terms of processing and storage capacity and takes long periods of time to converge.

In order to overcome the limitations of conventional numerical solvers, a fully data-driven hybrid physics-based Reduced-Order Model (ROM) that combines modal decomposition methods and Machine Learning (ML) has been developed. This methodology combines Higher-Order Singular Value Decomposition (HOSVD) with fully connected neural networks (FCNN) to enhance the spatial resolution of a fluid dynamics database. HOSVD has demonstrated its capability of extracting the main patterns and structures of fluid dynamics phenomena while denoising and compressing the database, while Neural Networks are able to model the non-linearities. This framework has been developed using Python’s ML libraries Tensorflow/Keras

Code in progress. Coming soon…

Temporal Forecasting

Purely Deep Learning Models

Forecasting models for fluid dynamics that rely entirely on deep learning are typically built using convolutional neural networks (CNNs). This is because datasets describing flow dynamics consist of two-dimensional or three-dimensional snapshots that encode the spatiotemporal characteristics of the flow. In this context, an autoencoder, where both the encoder and decoder are implemented with CNNs, enables efficient processing of the spatial dimensions of the dataset.

Specifically, the encoder constructs a latent representation that may capture key flow structures. Forecasting is then performed within this latent space, commonly using a convolutional long short-term memory (ConvLSTM) network. The predicted latent variables are subsequently passed through the decoder to reconstruct the snapshots corresponding to future flow dynamics.

Forecasting can be broadly categorized into two main types: point forecasting and probabilistic forecasting. Point forecasting approximates a deterministic function that describes the prediction process, whereas probabilistic forecasting estimates the underlying probability distribution of the forecast. A Residual Autoencoder is employed for point forecasting, while a Variational Autoencoder (VAE) is used for probabilistic forecasting.

Both the Residual Autoencoder and the VAE are autoregressive and they can process datasets in a five-dimensional tensor format, structured as [C,X,Y,Z,T]. Here, C denotes the dataset components, such as different velocity components, Reynolds numbers, or initial conditions. The indices X, Y and Z correspond to the spatial discretization along the x-, y- and z-axis, respectively, while T represents the temporal dimension, indicating the number of snapshots.

Unlike the POD-DL model, the architectures of these models are inherently dependent on the shape of the input snapshots. For the codes available on this page, the dataset must adhere to the following dimensional constraints: C = 3, X = 100, Y = 40 and Z = 64, with T remaining variable. Datasets with higher or lower dimensionality, or with a different arrangement of dimensions, may result in errors when executing the codes.

For example these codes can be tested with the dataset corresponding to the two-dimensional laminar flow past a cylinder, which is available here. Also note these models require a GPU for training.

Residual Autoencoder (point forecasting)

R. Abadía-Heredia, A.Corrochano, M. López-Martín and S. Le Clainche, “Generalization capabilities and robustness of hybrid machine learning models grounded in flow physics compared to purely deep learning models,” arXiv preprint arXiv:2404.17884 (2024).

Download the code here or open it in Colab.

Variational Autoencoder (probabilistic forecasting)

R. Abadía-Heredia, A.Corrochano, M. López-Martín and S. Le Clainche, “Generalization capabilities and robustness of hybrid machine learning models grounded in flow physics compared to purely deep learning models,” arXiv preprint arXiv:2404.17884 (2024).

Download the code here or open it in Colab.

Hybrid Predictive Model: POD-DL

The POD-DL model is a hybrid approach that integrates proper orthogonal decomposition (POD) with deep learning (DL) architectures, primarily an LSTM, to forecast the temporal evolution of flow dynamics. It follows an encoder-decoder structure, where the encoder corresponds to the matrix decomposition performed by POD, encapsulating the POD modes and coefficients. Since the temporal dynamics are encoded in the POD coefficients, the DL architecture focuses solely on forecasting these coefficients, while the POD modes remain unchanged. Once the new coefficients are computed, the decoder reconstructs the flow field by reversing the POD decomposition, i.e., performing a matrix multiplication. This process yields snapshots corresponding to future flow dynamics.

The forecasting can be conducted using either a fixed temporal horizon, where the model predicts a predetermined number of snapshots simultaneously, or an autoregressive approach, where multiple predictions are iteratively computed by feeding previously generated outputs back into the model.

The POD-DL model can process datasets represented as tensors of either four or five dimensions, structured as [C,X,Y,Z,T]. Here, C denotes the dataset components, such as different velocity components, Reynolds numbers, or initial conditions. The indices X, Y and Z correspond to the spatial discretization along the x-, y- and z-axis, respectively, while T represents the temporal dimension, indicating the number of snapshots. For four-dimensional tensors, the Z index may either be absent or set to Z = 1. Datasets with higher or lower dimensionality, or with a different arrangement of dimensions, may result in errors when executing the codes.

For example these codes can be tested with either the dataset corresponding to the two-dimensional laminar flow past a cylinder, which is available here, or the three-dimensional laminar flow past a cylinder, which is available here. Also note these models require a CPU for training.

POD-DL: Fixed temporal horizon

R. Abadía-Heredia, M. López-Martín, B. Carro, J. Arribas, J. Pérez, and S. Le Clainche, “A predictive hybrid reduced order model based on proper orthogonal decomposition combined with deep learning architectures,” Expert Systems with Applications 187, 115910 (2022).

Download the code here or open it in Colab.

POD-DL: Autoregressive

R. Abadía-Heredia, A.Corrochano, M. López-Martín and S. Le Clainche, “Generalization capabilities and robustness of hybrid machine learning models grounded in flow physics compared to purely deep learning models,” arXiv preprint arXiv:2404.17884 (2024).

Download the code here or open it in Colab.

Hybrid Predictive Model: HOSVD-DL

This model presents a hybrid ROM integrating Higher Order Singular Value Decomposition (HOSVD) with Long Short Term Memory (LSTM) architecture for temporal predictions. The integration of HOSVD enables dimensionality reduction and the LSTM model processes the sequences, capturing nonlinear dynamics and preserving long-term temporal correlations. Predictions are generated autoregressively, where forecasted solutions are fed back as inputs to itteratively predict the next step. While this approach can also be implemented with Singular Value Decomposition (SVD), HOSVD extends the capabilities of SVD by better preserving multi-dimensional structures, leading to improved performance in complex dynamical systems. This framework is implemented using TensorFlow/Keras for 4D and 5D tensors.

HOSVD-DL: Autoregressive

Code in progress. Coming soon…

Remote Sensing and DLinear

LC-SVD-DLinear (and LC-HOSVD-DLinear) are two hybrid machine learning models that combine low-cost singular value decomposition (LC-SVD) and low-cost high-order singular value decomposition with the DLinear architecture to efficiently forecast high-resolution experimental data collected from optimally placed sensors. These models operate by first using LC-SVD or LC-HOSVD to break down the experimental data and up-sample the temporal coefficients, POD modes, and singular values, significantly reducing computational cost while retaining all physical information. The reconstructed temporal coefficients are then processed by the DLinear model, which decomposes them into trend and seasonality components to identify temporal patterns and predict future values autoregressively, using the sliding window mechanism. Finally, the predicted temporal coefficients are combined with the reconstructed POD modes and singular values to generate high-resolution forecast snapshots, ensuring both accuracy and efficiency in handling complex, high-dimensional data.

Hetherington, A., Leonés, J. L., & Clainche, S. L. (2024). LC-SVD-DLinear: A low-cost physics-based hybrid machine learning model for data forecasting using sparse measurements. arXiv preprint arXiv:2411.17433.

Code in progress. Coming soon…

Control

Work in progress. Coming soon…