Three-Dimensional Field-Scale Coupled Thermo-Hydro-Mechanical Modeling: Parallel Computing Implementation

Abstract

An approach for the simulation of three-dimensional field-scale coupled thermo-hydro-mechanical problems is presented, including the implementation of parallel computation algorithms. The approach is designed to allow three-dimensional large-scale coupled simulations to be undertaken in reduced time. Owing to progress in computer technology, existing parallel implementations have been found to be ineffective, with the time taken for communication dominating any reduction in time gained by splitting computation across processors. After analysis of the behavior of the solver and the architecture of multicore, nodal, parallel computers, modification of the parallel algorithm using a novel hybrid message passing interface/open multiprocessing (MPI/OpenMP) method was implemented and found to yield significant improvements by reducing the amount of communication required. This finding reflects recent enhancements of current highperformance computing architectures. An increase in performance of 500% over existing parallel implementations on current processors was achieved for the solver. An example problem involving the Prototype Repository experiment undertaken by the Swedish Nuclear Fuel and Waste Management Co. [Svensk Kärnbränslehantering AB (SKB)] in Äspö, Sweden, has been presented to demonstrate situations in which parallel computation is invaluable because of the complex, highly coupled nature of the problem.

Coupled thermo-hydro-mechanical modelling: A new parallel approach

“…The System Matrix Build section has been previously efficiently parallelised as described in [1]. Hence, in this paper the focus of the work presented is on the algorithms related to the iterative solver.…”

“…More recently and increasingly with multicore processors, HPC systems are made up of nodes, which have multiple processing units with access to shared memory, attached together to form a larger HPC system; this configuration is sometimes termed a hybrid architecture. As described in [1] a previous parallelisation of a THM code resulted in significant time saving on a 600MHz MIPs R14000 system utilising an Ethernet interconnect. When executed on a 3.0GHz Intel Dual Core Xeon 5160 based HPC system with an InfiniBand interconnect, one section of the code, the easily parallelisable matrix build section, still performed well yielding a 4.8 times speed-up on 8 processing cores.…”

Reactive Transport of Chemicals in Compacted Bentonite under Nonisothermal Water Infiltration

“…The numerical investigation presented was carried out by extending the capabilities and theoretical aspects of a coupled thermal, hydraulic, chemical, and mechanical model (THCM) (COMPASS) (e.g., Thomas and He 1997;Seetharam et al 2007;Vardon et al 2011;Sedighi et al 2016) through (1) development of a theoretical formulation for multicomponent chemical transport under coupled thermohydraulic conditions (Sedighi et al 2011;;…”

Nonisothermal Multicomponent Reactive Transport Model for Unsaturated Soil

“…The shared-memory/multithreaded model usually outperforms the distributed-memory/message-passing model by virtue of avoiding data communication, but is restricted by the amount of processing cores available, on most HPC machines. It is shown by Vardon et al (2008) that on modern HPC machines using message-passing to parallelize the Bi-CG solver yielded only limited computational gain and communication times dominated the solver operation, especially when using more than 8 processing cores.…”

“…Up to a six times speed-up has been found when using 32 processing cores for problems consisting of approximately 500,000 degrees of freedom. More details and a comprehensive analysis of the computational performance can be found in Vardon et al (2011).…”