A GPU-accelerated package for simulation of flow in nanoporous source rocks with many-body dissipative particle dynamics

Abstract

Mesoscopic simulations of hydrocarbon flow in source shales are challenging, in part due to the heterogeneous shale pores with sizes ranging from a few nanometers to a few micrometers. Additionally, the sub-continuum fluid-fluid and fluid-solid interactions in nano-to micro-scale shale pores, which are physically and chemically sophisticated, must be captured. To address those challenges, we present a GPU-accelerated package for simulation of flow in nano-to micro-pore networks with a many-body dissipative particle dynamics (mDPD) mesoscale model. Based on a fully distributed parallel paradigm, the code offloads all intensive workloads on GPUs. Other advancements, such as smart particle packing and no-slip boundary condition in complex pore geometries, are also implemented for the construction and the simulation of the realistic shale pores from 3D nanometer-resolution stack images. Our code is validated for accuracy and compared against the CPU counterpart for speedup. In our benchmark tests, the code delivers nearly perfect strong scaling and weak scaling (with up to 512 million particles) on up to 512 K20X GPUs on Oak Ridge National Laboratory's (ORNL) Titan supercomputer. Moreover, a single-GPU benchmark on ORNL's SummitDev and IBM's AC922 suggests that the host-to-device NVLink can boost performance over PCIe by a remarkable 40%. Lastly, we demonstrate, through a flow simulation in realistic shale pores, that the CPU counterpart requires 840 Power9 cores to rival the performance delivered by our package with four V100 GPUs on ORNL's Summit architecture. This simulation package enables quick-turnaround and high-throughput mesoscopic numerical simulations for investigating complex flow phenomena in nano-to micro-porous rocks with realistic pore geometries. Program summaryProgram title: USER MESO 2.5 Licensing provisions: GNU General Public License 3 Programming language: CUDA C/C++ with MPI and OpenMP Nature of problem: Particle-based simulation of multiphase flow and fluid-solid interaction in nano-to micro-scale pore networks of arbitrary pore geometries. Solution method: Fluid particles and solid wall particles are modeled with a many-body dissipative particle dynamics (mDPD) model -a mesoscopic model for coarse-grained fluid and solid molecules. The pore surface wall boundary for arbitrary surface geometries is modeled with a no-slip boundary condition for fluid particles that prevents fluid particles from indefinitely penetrating in the walls. The time evolution of the system is integrated using the Velocity-Verlet algorithm. Restrictions: The code is compatible with NVIDIA GPUs with compute capability 3.0 and above. Unusual features: The code is implemented on GPGPUs with significantly improved speed.Recently we developed an mDPD based nano to micro-scale pore flow model and applied it for multiphase flow simulations in source shale [58]. In that model, realistic shale pore geometries are constructed based on 3D voxel 2/21

Confinement Effect on Porosity and Permeability of Shales

“…Depending on the complexity of the model and/or the objectives of the studied problem, the scale and features of the simulated model may vary from nanometers to centimeters. Various computational models, such as molecular dynamics (MD) 22,23 , direct simulation Monte Carlo (DSMC) 24 , dissipative particle dynamics (DPD) 25,26 , Lattice Boltzmann (LB) 27,28 , and many continuum-flow-based models, have been used for simulations of fluid flow in shales to calculate their permeability. The main deference between these simulations lies in the size of the modeled system and the type of fluid flow they are capable of modeling.…”

Nanofabrication of synthetic nanoporous geomaterials: from nanoscale-resolution 3D imaging to nano-3D-printed digital (shale) rock

“…www.nature.com/scientificreports/ fluid flow and transport phenomena, motivates the need for developing novel geo-architected materials suited for more sophisticated experiments to validate fluid flow models recently developed by numerus research groups [14][15][16][17] . There is considerable evidence that the known laws of adsorption, reaction, phase transitions, and flow behavior are affected by the presence of fluids confined in porous materials with nanometer-sized pores 18,19 .…”

Adaptive neural network-based approximation to accelerate eulerian fluid simulation

“…We use the average of Q loss in the k pairs as the predicted Q loss in the current time interval. In our evaluation, we use different values for k, but find that k ∈ [4,6] is usually sufficient to give accurate prediction, hence we choose k = 4 to reduce runtime overhead.…”

“…The fluid simulation aims to study the flow of fluid materials and has been widely applied to multiple disciplines such as chemical physics and material science [1][2][3]. However, the simulation of fluid dynamics usually requires prohibitively high computational resources [4,5] and thus limits its application in the related fields.…”