Transformer-based deep learning models for predicting permeability of porous media

Abstract

The direct acquisition of the permeability of porous media by digital images helps to enhance our understanding of and facilitate research into the problem of subsurface flow. A complex pore space makes the numerical simulation methods used to calculate the permeability quite time-consuming. Deep learning models represented by three-dimensional convolutional neural networks (3D CNNs), as a promising approach to improving efficiency, have made significant advances concerning predicting the permeability of porous media. However, 3D CNNs only have the ability to represent the local information of 3D images, and they cannot consider the spatial correlation between 2D slices, a significant factor in the reconstruction of porous media. This study combines a 2D CNN and a self-attention mechanism to propose a novel CNN-Transformer hybrid neural network that can make full use of the 2D slice sequences of porous media to accurately predict their permeability. In addition, we added physical information to the slice sequences and built a PhyCNN-Transformer model to reflect the impact of physical properties on permeability prediction. In terms of dataset preparation, we used the publicly available DeePore porous media dataset with the labeled permeability calculated by pore network modelling (PNM). We compared the two transformer-based models with a 3D CNN in terms of parameter number, training efficiency, prediction performance, and generalization, and the results showed significant improvement. Combined with the transfer learning method, we demonstrate the superior generalization ability of the transformer-based models to unfamiliar samples with small sample sizes.

Computational fluid dynamics study on three‐dimensional polymeric scaffolds to predict wall shear stress using machine learning models for bone tissue engineering applications

“…On the other hand, WSS levels exceeding 60 mPa are associated with cell death. [8][9][10][11] Laminar flow CFD studies investigated on biologically designed scaffolds show improved biological behavior than rectangular scaffolds. Biologically designed scaffolds have uniform pore sizes and complex architecture.…”

“…A hybrid neural network that combines CNN and transformer models has been developed to accurately predict permeability based on two-dimensional slice sequences of the porous media. 11 Scaffold with a non-uniform flow path and interlocked pores provides a maximum number of effective stem cell accommodation. 13,14 This study aims to modify the design of rectangular scaffolds by following a triangular pitch pattern that offers interlock pores, maximum cell accommodation, and better biological performance.…”

“…When WSS falls between 0 and 30 mPa, it fosters the overall biological function of stem cells. On the other hand, WSS levels exceeding 60 mPa are associated with cell death 8–11 . Laminar flow CFD studies investigated on biologically designed scaffolds show improved biological behavior than rectangular scaffolds.…”

“…The use of numerical simulation methods to calculate the permeability of porous media can be quite time‐consuming due to the complexity of their pore space. A hybrid neural network that combines CNN and transformer models has been developed to accurately predict permeability based on two‐dimensional slice sequences of the porous media 11 …”