Modeling three-dimensional (3D) turbulence by neural networks is difcult because 3D turbulence is highly-nonlinear with high degrees of freedom and the corresponding simulation is memoryintensive. Recently, the attention mechanism has been shown as a promising approach to boost the performance of neural networks on turbulence simulation. However, the standard self-attention mechanism uses O(n2) time and space with respect to input dimension n, and such quadratic complexity has become the main bottleneck for attention to be applied on 3D turbulence simulation. In this work, we resolve this issue with the concept of linear attention network. The linear attention approximates the standard attention by adding two linear projections, reducing the overall self-attention complexity from O(n2) to O(n) in both time and space. The linear attention coupled Fourier neural operator (LAFNO) is developed for the simulation of 3D isotropic turbulence and free shear turbulence. Numerical simulations show that the linear attention mechanism provides 40% error reduction at the same level of computational cost, and LAFNO can accurately reconstruct a variety of statistics and instantaneous spatial structures of 3D turbulence. The linear attention method would be helpful for the improvement of neural network models of 3D nonlinear problems involving high-dimensional data in other scientifc domains.

Long-term predictions of nonlinear dynamics of three-dimensional (3D) turbulence are very challenging for machine learning approaches. In this paper, we propose an implicit U-Net enhanced Fourier neural operator (IU-FNO) for stable and efficient predictions on the long-term large-scale dynamics of turbulence. The IU-FNO model employs implicit recurrent Fourier layers for deeper network extension and incorporates the U-net network for the accurate prediction on small-scale flow structures. The model is systematically tested in large-eddy simulations of three types of 3D turbulence, including forced homogeneous isotropic turbulence, temporally evolving turbulent mixing layer, and decaying homogeneous isotropic turbulence. The numerical simulations demonstrate that the IU-FNO model is more accurate than other FNO-based models, including vanilla FNO, implicit FNO (IFNO), and U-Net enhanced FNO (U-FNO), and dynamic Smagorinsky model (DSM) in predicting a variety of statistics, including the velocity spectrum, probability density functions of vorticity and velocity increments, and instantaneous spatial structures of flow field. Moreover, IU-FNO improves long-term stable predictions, which has not been achieved by the previous versions of FNO. Moreover, the proposed model is much faster than traditional large-eddy simulation with the DSM model and can be well generalized to the situations of higher Taylor–Reynolds numbers and unseen flow regime of decaying turbulence.