Overview

This tutorial explains how to use the generalized adaptive CFD–POD–LSTM pipeline developed at ETSIAE–UPM. The pipeline couples an OpenFOAM CFD solver with a data-driven surrogate model (Proper Orthogonal Decomposition + Long Short-Term Memory network) in a closed loop. At each iteration, OpenFOAM generates a block of flow snapshots, the LSTM is trained on them and predicts the next block, and the solver restarts — keeping prediction accuracy controlled over arbitrarily long horizons.

The generalization introduced in this work replaces all scattered shell and Python configuration files from the original implementation (Zou 2025) with a single config.yaml file and a single Python orchestrator (directorpython.py), making the pipeline portable to any OpenFOAM case.

Code

Comming soon…

What This Tutorial Covers

• What CFD adaptive prediction is and why it is needed.
• How the CFD–POD–LSTM loop works (adaptive framework).
• How POD via truncated SVD reduces the problem dimensionality.
• How the LSTM is trained and used for auto-regressive prediction.
• How to install the required Python environment.
• How to copy the pipeline into any OpenFOAM case.
• How to configure every parameter in config.yaml.
• How to run in offline mode (SOLO_PYTHON=true) on pre-computed snapshots.
• How to run the full adaptive loop (SOLO_PYTHON=false) with a live OpenFOAM simulation.
• How to read the output logs and visualise results in ParaView.

Full Tutorial Content

Table of Contents

1. Introduction
2. The adaptive framework
3. Methodology: POD + LSTM
4. File structure
5. Environment setup
6. config.yaml — all parameters
7. Run the pipeline
8. Summary / Checklist

1. Introduction

The challenge of high-fidelity simulations

Computational Fluid Dynamics (CFD) solvers — such as OpenFOAM — numerically solve the Navier–Stokes equations to predict how a fluid (air, water, etc.) moves through a geometry. To get accurate results, the solver must resolve all relevant spatial and temporal scales of the flow, which translates directly into very fine meshes and very small time steps.

This accuracy comes at a steep computational price:

• A single simulation of a realistic case can take hours or days on a cluster.
• Engineering tasks like parameter optimisation or uncertainty quantification require thousands of such evaluations.
• Running all of them with a full CFD solver is often completely infeasible.

Why machine learning?

Data-driven surrogate models can learn the dynamics of a flow from a set of precomputed snapshots and then predict future states in a fraction of the time. The idea is to let the CFD solver run for a while, collect its output, train a neural network, and then let the network take over prediction — bypassing the expensive solver.

The key challenge is prediction drift: purely data-driven models are trained once and their accuracy degrades over time as the flow evolves and the model encounters conditions not seen during training.

Adaptive prediction: the solution

Adaptive prediction solves the drift problem by coupling the ML model with the live CFD solver in a closed loop. Instead of training once and predicting forever, the pipeline alternates between:

1. CFD solver — generates new, accurate snapshots for a window of time.
2. ML model (POD-DL) — trains on those snapshots and predicts the next window.
3. Repeat — the solver restarts from near the end of the ML prediction, corrects any drift, and the cycle continues.

This way, accuracy remains controlled over arbitrarily long prediction horizons and the model stays robust to shifts in the flow regime.

The only file you need to edit is config.yaml. The only command you need to run is python pyPseudo_Adaptive_parallel/directorpython.py.

2. The adaptive framework

How the loop works

The pipeline alternates between blocks of CFD simulation and blocks of ML prediction:

At each iteration:

1. OpenFOAM runs from T₀ to T₁, saving a snapshot of the flow field every TIME_STEP_WRITE_OF seconds.
2. POD-DL reads those snapshots, applies truncated SVD to reduce them to a low-dimensional representation, and trains an LSTM on the resulting temporal coefficients.
3. LSTM predicts the coefficients from T₁ to T₂ in an auto-regressive loop; the full flow field is reconstructed from those coefficients.
4. OpenFOAM restarts from T₂ − TIME_AHEAD (a small overlap ensures continuity between the CFD and ML blocks).
5. Steps 1–4 repeat NUM_ITERATIONS times.

Key design choices

3. Methodology: POD + LSTM

The big picture

The full pipeline goes from raw CFD snapshots to predicted flow fields in five steps:

Why dimensionality reduction?

A typical CFD mesh has millions of cells and each field (velocity, pressure) is a vector at every cell. Feeding raw snapshots directly into a neural network would be computationally prohibitive. Proper Orthogonal Decomposition (POD) compresses the data: instead of predicting millions of numbers, the LSTM only needs to predict a handful of temporal coefficients.

POD via truncated SVD

The snapshot matrix X (one column per time step, one row per mesh cell) is factorised via Singular Value Decomposition (SVD). We then keep only the r₀ most energetic modes — this is the “truncation” step. The result is a small set of temporal coefficients that capture the essential dynamics of the flow:

• U_r — spatial POD modes (basis vectors): the “shapes” of the flow patterns
• Σ_r — singular values: how much energy each mode carries
• C — temporal coefficients: how much each mode contributes at each time step

The reconstruction error (RRMSE) tells you how much information is lost by keeping only r₀ modes. A good choice of NUM_MODES (r₀) keeps this below ~5 %.

LSTM prediction

The LSTM receives a sliding window of past temporal coefficients and predicts the next one. This prediction is then fed back as input to predict the following step — and so on, auto-regressively, for NUM_PREDS steps:

Once the LSTM has predicted all the future temporal coefficients, the full flow field is reconstructed by multiplying back with the spatial modes (U_r · Σ_r · Ĉ).

Training

The LSTM is trained to minimise the Mean Squared Error (MSE) between its predicted coefficients and the true ones from the CFD data. Two key features make training efficient:

• Transfer learning: if a model checkpoint exists from the previous adaptive iteration, it is loaded and fine-tuned instead of training from scratch. This makes each successive round converge much faster.
• Early stopping: training halts automatically when loss drops below LOSS_THRESHOLD.

The two hyperparameters with the greatest impact on prediction quality are NUM_MODES (r₀) and INP_SEQ (L). Start with NUM_MODES=5 and INP_SEQ=8 and tune from there.

Divergence monitoring: CE and TE

As the LSTM predicts further into the future, its error grows. The pipeline monitors this error in real time using two metrics:

• Consistency Estimate (CE) — how far the LSTM prediction has drifted from the POD-reconstructed reference.
• Truncation Estimate (TE) — how much information is lost by retaining only r₀ modes.

When either metric exceeds its threshold (CE_THRESHOLD or TE_THRESHOLD in config.yaml), the CFD solver is automatically restarted to correct the drift:

In fixed-interval mode both thresholds are set to 10⁹ (effectively disabled) and the solver restarts after a fixed number of predicted snapshots.

Divergence monitoring: Mahalanobis distance and ensemble UQ

Two additional, more sophisticated criteria are available:

Mahalanobis distance — measures how far a predicted coefficient vector has strayed from the training distribution. If the prediction lands in a region of the coefficient space that the model has never seen during training, the solver is recalled. This is controlled by the mahalanobis_thr parameter in config.yaml.

Ensemble uncertainty quantification (UQ) — instead of training a single LSTM, an ensemble of M models is trained. At each prediction step, each member produces a slightly different output. The spread (standard deviation) across ensemble members is a direct measure of prediction uncertainty:

When the uncertainty exceeds a user-defined threshold (sigma_thr in config.yaml), the solver is recalled.

4. File structure

What you need inside your OpenFOAM case directory

my_openfoam_case/
│
├── 0/                          ← OpenFOAM initial conditions (U, p, …)
├── constant/                   ← mesh + fluid properties (unchanged)
├── system/
│   ├── controlDict             ← auto-modified by directorpython.py — do not set startTime/endTime manually
│   └── decomposeParDict        ← numberOfSubdomains and n-tuple auto-updated from config.yaml
│   └── ...
│
├── pyPseudo_Adaptive_parallel/ ← copy this folder from the repository
│   ├── directorpython.py       ← Python orchestrator: reads config.yaml and drives the full loop
│   ├── field_read_write.py     ← reads and writes OpenFOAM binary and ASCII fields
│   ├── load_data.py            ← snapshot loading and preprocessing
│   ├── forecasting_and_uq.py   ← SVD, LSTM training, prediction, and UQ metrics
│   ├── Forecasting/
│   │   ├── nn_lstm.py          ← LSTM architecture definition (PyTorch)
│   │   └── model_trainer.py    ← training loop and checkpointing logic
│   └── training_inference_module.py  ← top-level training / inference interface
│
└── config.yaml                 ← the only file you need to edit

How directorpython.py drives the loop:

For each iteration it automatically:

1. Reads config.yaml
2. Auto-detects OpenFOAM (or uses ENV_OF if set)
3. Updates system/controlDict (startTime, endTime, writeInterval, writeFormat) via foamDictionary
4. Runs mpirun -np N <solver> -parallel and writes the log to log_OF_<start>_<end>.txt
5. Runs reconstructPar and deletes processor*/ folders
6. Calls the ML module with the parameters from config.yaml
7. Goes back to step 3 for the next iteration

You never need to touch any of these files.

5. Environment setup

5.1 Create a Python virtual environment

You need Python 3.10 (or 3.9+). Choose either venv or conda:

# Option A — venv (recommended, no extra install needed)
python3 -m venv ~/envs/cfd_ml
source ~/envs/cfd_ml/bin/activate

# Option B — conda
conda create -n cfd_ml python=3.10 -y
conda activate cfd_ml

5.2 Install dependencies

pip install torch numpy scipy matplotlib pyyaml psutil

Verify that everything installed correctly:

python -c "import torch, yaml, psutil; print('OK')"
# Expected output: OK

5.3 Point config.yaml to your environment

In config.yaml, under parametros_entorno, set ENV_CONDA to the full path of your Python binary:

parametros_entorno:
  ENV_CONDA: "/home/youruser/envs/cfd_ml/bin/python"               # venv
  # ENV_CONDA: "/home/w460/youruser/.conda/envs/cfd_ml/bin/python" # Magerit conda
  # ENV_CONDA: "NULL"                                              # use system python3

directorpython.py auto-detects OpenFOAM by searching /opt/openfoam*/etc/bashrc. If your installation is elsewhere, set the full path:

  ENV_OF: "/media/apps/avx512-2021/software/OpenFOAM/12-foss-2023a/OpenFOAM-12/etc/bashrc"
  # ENV_OF: "NULL"   # auto-detect

6. config.yaml — all parameters

The entire experiment is controlled from a single YAML file. Below is every section and every parameter.

Overview

All parameters live here. You never need to open any .py file.

6.1 parametros_entorno — Environment

parametros_entorno:
  ENV_OF: "NULL"                                   # OpenFOAM bashrc path; NULL = auto-detect
  ENV_CONDA: "/home/user/envs/cfd_ml/bin/python"   # path to your Python binary
  N_PROCESOS: 4                                    # MPI cores for OpenFOAM
  DECOMP_N: [2, 2, 1]                              # domain decomposition: n1×n2×n3 must equal N_PROCESOS
  SOLO_PYTHON: false                               # true = ML only; false = full OF + ML loop

6.2 parametros_flujo — Timing

parametros_flujo:
  INITIAL_TIME: 100          # start time of the first CFD block
  TIME_PERIOD_OF: 100        # duration of each OpenFOAM window (seconds)
  TIME_PERIOD_PY: 100        # duration of each ML prediction window (seconds)
  TIME_STEP_WRITE_OF: 1      # OpenFOAM writeInterval (seconds)
  TIME_STEP_WRITE_PY: 1      # time resolution of ML predictions (seconds)
  TIME_AHEAD: 5              # OF–ML overlap (seconds); typically 5 × TIME_STEP_WRITE_OF
  NUM_ITERATIONS: 2          # number of OF + ML rounds

  # Only used when SOLO_PYTHON: true
  START_TIME_SP: --         # first available snapshot
  MID_TIME_SP: --           # end of training data
  END_TIME_SP: --           # end of prediction

6.3 parametros_caso — Case

parametros_caso:
  CASE_NAME: "cylinder"    # used to name saved model files
  VARIABLES:
    - "U"
    - "p"

6.4 parametros_svd — Dimensionality reduction

parametros_svd:
  SVD_MODEL: "SVD"       # "SVD" (standard) or "lcSVD" (low-cost, sensor-based)
  NUM_MODES: 5           # number of POD modes r₀ to retain
  NUM_SENSORS: --        # sensor points (lcSVD only)
  RRMSE_CRITI: --        # acceptable RRMSE % for sensor selection (lcSVD only)
  OVERLAP_POINTS: --    # spatial overlap between parallel processing chunks

6.5 parametros_lstm — LSTM architecture and training

parametros_lstm:
  MODEL_NAME: "lstm"
  HIDDEN_SIZE: 100         # hidden units in the LSTM cell
  NUM_LAYERS: 1            # stacked LSTM layers
  INP_SEQ: 8               # input window length (past time steps fed to the model)
  OUT_SEQ: 1               # steps predicted at once (always keep at 1)
  STRIDE: 0
  STEP: 1
  VAL_SIZE: 0
  EPOCHS: 1000             # maximum training epochs
  LOSS_THRESHOLD: 1.0e-8   # stop early if loss drops below this value
  BATCH_SIZE: 4
  OPTIMIZER_NAME: "Adam"
  OPTIMIZER_HPARAMS:
    lr: 1.0e-3
    weight_decay: 1.0e-4
  VERSION: 0               # checkpoint version index
  SNAP_STEP: 1.0
  NUM_PREDS: 100            # number of auto-regressive predictions per ML round

6.6 parametros_umbrales — Thresholds (CE / TE)

parametros_umbrales:
  CE_THRESHOLD: 1.0e+9   # consistency estimate threshold
  TE_THRESHOLD: 1.0e+9   # truncation estimate threshold

When set to 1e9 (the default), both thresholds are effectively disabled and the pipeline uses fixed time windows (TIME_PERIOD_PY). To enable automatic early restart of OpenFOAM when prediction quality degrades, set these to meaningful values based on your flow (consult Zou 2025 for derivation).

6.7 parametros_rutas — Paths

parametros_rutas:
  CFD_DIR: "/path/to/my_openfoam_case"
  SAVED_MODELS_DIR: "./pyPseudo_Adaptive_parallel"
  SENSORS_DIR: "./pyPseudo_Adaptive_parallel/sensors"

7. Run the pipeline

Two modes are available. Choose based on whether you already have CFD snapshots.

7.1 SOLO_PYTHON mode — offline prediction on pre-computed snapshots

Use this when you already have all the OpenFOAM time folders and want to test the ML module without running the solver.

Step 1 — Enable SOLO_PYTHON and define the time split

parametros_entorno:
  SOLO_PYTHON: false

parametros_flujo:
  START_TIME_SP: 100    # first available time folder
  MID_TIME_SP: 200      # training uses [START → MID]
  END_TIME_SP: 300      # ML predicts [MID → END]

Step 2 — Point to the data

parametros_rutas:
  CFD_DIR: "/path/to/my_case_snapshots"   # must contain 100/, 101/, 102/, …

Step 3 — Run

cd /path/to/my_openfoam_case/
python pyPseudo_Adaptive_parallel/directorpython.py

Log created automatically:

log_py_100_200.txt   ← epoch / loss per iteration + RRMSE of reconstruction

7.2 Full adaptive loop — OpenFOAM + ML

Use this for a live simulation where the CFD solver and the ML model alternate.

Step 1 — Configure the timing

parametros_entorno:
  SOLO_PYTHON: false
  N_PROCESOS: 48             # directorpython.py updates numberOfSubdomains automatically
  DECOMP_N: [6, 4, 2]        # 6×4×2 = 48 — updates hierarchicalCoeffs/n automatically

parametros_flujo:
  INITIAL_TIME: 100
  TIME_PERIOD_OF: 100        # OF runs for 100 s = 100 snaps at dt=1
  TIME_PERIOD_PY: 100        # ML predicts 100 s = 100 snaps
  TIME_STEP_WRITE_OF: 1
  TIME_AHEAD: 5              # 5-snapshot overlap
  NUM_ITERATIONS: 2

Step 2 — Run

cd /path/to/my_openfoam_case/
python pyPseudo_Adaptive_parallel/directorpython.py

Expected log files (for NUM_ITERATIONS=2):

log_OF_100_200.txt         ← OpenFOAM round 1
log_py_200_300.txt         ← ML round 1
log_OF_295_400.txt         ← OpenFOAM round 2 (restarts from T₁ − TIME_AHEAD)
log_py_400_500.txt         ← ML round 2

7.3 Output files and how to interpret them

After a successful run:

my_openfoam_case/
├── 100/, 101/, …, 200/                        ← CFD snapshots (OpenFOAM format)
├── 200/, 201/, …, 300/                        ← ML predictions (same OpenFOAM format)
├── log_OF_100_200.txt                         ← OpenFOAM residuals log
├── log_py_200_300.txt                         ← ML training + prediction log
└── pyPseudo_Adaptive_parallel/
    └── cylinder_1_100_100_5_saved_models/
        ├── best_model_z_0.pt                  ← LSTM weights for field U
        └── best_model_p_0.pt                  ← LSTM weights for field p

Reading the logs:

• log_OF_*.txt — standard OpenFOAM log: PISO residuals, continuity errors, same as any standalone OpenFOAM run.
• log_py_*.txt — ML log: epoch number, MSE loss, and SVD reconstruction RRMSE for each variable. A healthy run shows steadily decreasing loss and RRMSE below ~5%.
• Success: last line reads "All iterations completed successfully!" — everything worked.
• ERROR: check ENV_OF (OpenFOAM not found) or ENV_CONDA (Python not found), and confirm your OpenFOAM case runs correctly on its own before using the pipeline.

Visualising results in ParaView:

The ML prediction folders are standard OpenFOAM time folders. Open them alongside the CFD folders:

ParaView → File > Open → select your .foam file
         → Apply → scrub through time → select field (U, p, …)

Both CFD and ML time steps appear in the same timeline, letting you compare them directly.

8. Summary / Checklist

Use this checklist every time you set up a new experiment from scratch.

Step 1 — Install the environment (once per machine)

python3 -m venv ~/envs/cfd_ml
source ~/envs/cfd_ml/bin/activate
pip install torch numpy scipy matplotlib pyyaml psutil

python -c "import torch, yaml, psutil; print('OK')"

Step 2 — Copy the code into your OpenFOAM case

cp -r /path/to/repo/pyPseudo_Adaptive_parallel/  /path/to/my_openfoam_case/
cp    /path/to/repo/config.yaml                   /path/to/my_openfoam_case/

Step 3 — Edit config.yaml

These are the fields you must set before running:

Step 4 — Run

cd /path/to/my_openfoam_case/
python pyPseudo_Adaptive_parallel/directorpython.py

Step 5 — Verify

tail -1 log_py_*.txt
# Expected: "All iterations completed successfully!"

Then open ParaView, load your .foam file, and compare predicted vs CFD fields across the time timeline.

Summary: a single config.yaml and a single python command replace all the scattered shell scripts and Python files of the original implementation. The pipeline works with any OpenFOAM case — point CFD_DIR to your case, set your variables, and run.

Contributors

• Xiangrui Zou
• Carlos Sainz García
• Mikel Navarro Huarte

ModelFLOWs-UPM · Adaptive CFD-LSTM · POD/SVD + Deep Learning
pyPseudo_Adaptive_parallel/ + config.yaml
Based on:

• Abadía-Heredia, R., Zou., X., López-Martín, M., Le Clainche, S., An Adaptive Framework for Autoregressive Forecasting in CFD Using Hybrid Modal Decomposition and Deep Learning, arXiv:2505.01531, 2025
• Zou, X., Zhao, Z., Barragán, G., Le Clainche, S., Divergence-aware adaptive prediction framework for accelerating CFD simulations of unsteady flows, arXiv:2605.24150 , 2026.