Generalizability of transformer-based deep learning for multidimensional turbulent flow data

Predicting the large-scale dynamics of three-dimensional (3D) turbulence is challenging for machine learning approaches. This paper introduces a transformer-based neural operator (TNO) to achieve precise and efficient predictions in the large-eddy simulation (LES) of 3D turbulence. The performance of the proposed TNO model is systematically tested and compared with LES using classical sub-grid scale models, including the dynamic Smagorinsky model (DSM) and the dynamic mixed model (DMM), as well as the original Fourier neural operator (FNO) model, in homogeneous isotropic turbulence (HIT) and free-shear turbulent mixing layer. The numerical simulations comprehensively evaluate the performance of these models on a variety of flow statistics, including the velocity spectrum, the probability density functions (PDFs) of vorticity, the PDFs of velocity increments, the evolution of turbulent kinetic energy, and the iso-surface of the Q-criterion. The results indicate that the accuracy of the TNO model is comparable to the LES with DSM model and outperforms the FNO model and LES using DMM in HIT. In the free-shear turbulence, the TNO model exhibits superior accuracy compared to other models. Moreover, the TNO model has fewer parameters than the FNO model and enables long-term stable predictions, which the FNO model cannot achieve. The well-trained TNO model is significantly faster than traditional LES with DSM and DMM models and can be generalized to higher Taylor–Reynolds number cases, indicating its strong potential for 3D nonlinear engineering applications.

A transformer-based neural operator for large-eddy simulation of turbulence

Li,

Liu,

Peng

et al. 2024

Self-supervised transformers for turbulent flow time series

Drikakis,

Kokkinakis,

Fung

et al. 2024

There has been a rapid advancement in deep learning models for diverse research fields and, more recently, in fluid dynamics. This study presents self-supervised transformers' deep learning for complex turbulent flow signals across various test problems. Self-supervision aims to leverage the ability to extract meaningful representations from sparse flow time-series data to improve the transformer model accuracy and computational efficiency. Two high-speed flow cases are considered: a supersonic compression ramp and shock-boundary layer interaction over a statically deformed surface. Several training scenarios are investigated across the two different supersonic configurations. The training data concern wall pressure fluctuations due to their importance in aerodynamics, aeroelasticity, noise, and acoustic fatigue. The results provide insight into transformers, self-supervision, and deep learning with application to complex time series. The architecture is extendable to other research domains where time series data are essential.

The effects of hyperparameters on deep learning of turbulent signals

Tirchas,

Drikakis,

Kokkinakis

et al. 2024

The effect of hyperparameter selection in deep learning (DL) models for fluid dynamics remains an open question in the current scientific literature. Many authors report results using deep learning models. However, better insight is required to assess deep learning models' behavior, particularly for complex datasets such as turbulent signals. This study presents a meticulous investigation of the long short-term memory (LSTM) hyperparameters, focusing specifically on applications involving predicting signals in shock turbulent boundary layer interaction. Unlike conventional methodologies that utilize automated optimization techniques, this research explores the intricacies and impact of manual adjustments to the deep learning model. The investigation includes the number of layers, neurons per layer, learning rate, dropout rate, and batch size to investigate their impact on the model's predictive accuracy and computational efficiency. The paper details the iterative tuning process through a series of experimental setups, highlighting how each parameter adjustment contributes to a deeper understanding of complex, time-series data. The findings emphasize the effectiveness of precise manual tuning in achieving superior model performance, providing valuable insights to researchers and practitioners who seek to leverage long short-term memory networks for intricate temporal data analysis. The optimization not only refines the predictability of the long short-term memory in specific contexts but also serves as a guide for similar manual tuning in other specialized domains, thereby informing the development of more effective deep learning models.