Evaluating the performance of parallel subsurface simulators: An illustrative example with PFLOTRAN

Hammond, Glenn Edward; Lichtner, Peter C.; Mills, Richard Tran

doi:10.1002/2012wr013483

Cited by 289 publications

(217 citation statements)

References 48 publications

Supporting

Mentioning

214

Contrasting

Unclassified

Order By: Relevance

“…B6), which is nonphysical. One approach to avoid negative concentration is to scale back the update with a scaling factor (λ) (Bethke, 2007;Hammond, 2003) such that…”

Section: Scaling Clipping and Log Transformation For Avoiding Negatmentioning

confidence: 99%

“…A third approach, log transformation, also ensures a positive solution (Bethke, 2007;Hammond, 2003;Parkhurst and Appelo, 1999). It is widely used in geochemical codes for describing highly variable concentrations for primary species such as H + or O 2 that can vary over many orders of magnitude as pH or redox state changes without the need to use variable switching.…”

Section: Scaling Clipping and Log Transformation For Avoiding Negatmentioning

confidence: 99%

“…One is to use the logarithm concentration as the primary variable (Bethke, 2007;Hammond, 2003;Parkhurst and Appelo, 1999). The other two either scale back the update vector (Bethke, 2007;Hammond, 2003) or clip the concentrations for the species that are going negative (Yeh et al, 2004;White and McGrail, 2005;Sonnenthal et al, 2014) in each iteration. Except that log transformation is more computationally demanding (Hammond, 2003), how these methods for enforcing non-negativity affect computational accuracy and efficiency is rarely discussed.…”

Section: Introductionmentioning

confidence: 99%

“…The objective of this work is to explore some of the implementation issues associated with using RTM codes in LSMs, with the ultimate goal being accurate, efficient, robust, and configurable representations of subsurface biogeochemical reactions in CLM. To this end, we develop an alternative implementation of an existing CLM biogeochemical reaction network using PFLOTRAN (massively parallel subsurface flow and reactive transport) Hammond et al, 2014); couple that model to CLM (we refer to the coupled model as CLM-PFLOTRAN); test the implementation at Arctic, temperate, and tropical sites; and examine the implication of using scaling, clipping, and log transformation for avoiding negative concentrations. Although we focus on a number of carbon-nitrogen reactions implemented in PFLOTRAN and CLM, the critical numerical issue of avoiding negative concentrations has broader relevance as biogeochemical representations in LSMs become more mechanistic.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Addressing numerical challenges in introducing a reactive transport code into a land surface model: a biogeochemical modeling proof-of-concept with CLM–PFLOTRAN 1.0

et al. 2016

View full text Add to dashboard Cite

Abstract.We explore coupling to a configurable subsurface reactive transport code as a flexible and extensible approach to biogeochemistry in land surface models. A reaction network with the Community Land Model carbonnitrogen (CLM-CN) decomposition, nitrification, denitrification, and plant uptake is used as an example. We implement the reactions in the open-source PFLOTRAN (massively parallel subsurface flow and reactive transport) code and couple it with the CLM. To make the rate formulae designed for use in explicit time stepping in CLMs compatible with the implicit time stepping used in PFLOTRAN, the Monod substrate rate-limiting function with a residual concentration is used to represent the limitation of nitrogen availability on plant uptake and immobilization. We demonstrate that CLM-PFLOTRAN predictions (without invoking PFLOTRAN transport) are consistent with CLM4.5 for Arctic, temperate, and tropical sites.Switching from explicit to implicit method increases rigor but introduces numerical challenges. Care needs to be taken to use scaling, clipping, or log transformation to avoid negative concentrations during the Newton iterations. With a tight relative update tolerance (STOL) to avoid false convergence, an accurate solution can be achieved with about 50 % more computing time than CLM in point mode site simulations using either the scaling or clipping methods. The log transformation method takes 60-100 % more computing time than CLM. The computing time increases slightly for clipping and scaling; it increases substantially for log transformation for half saturation decrease from 10 −3 to 10 −9 mol m −3 , which normally results in decreasing nitrogen concentrations. The frequent occurrence of very low concentrations (e.g. below nanomolar) can increase the computing time for clipping or scaling by about 20 %, double for log transformation. Overall, the log transformation method is accurate and robust, and the clipping and scaling methods are efficient. When the reaction network is highly nonlinear or the half saturation or residual concentration is very low, the allowable time-step cuts may need to be increased for robustness for the log transformation method, or STOL may need to be tightened for the clipping and scaling methods to avoid false convergence.As some biogeochemical processes (e.g., methane and nitrous oxide reactions) involve very low half saturation and thresholds, this work provides insights for addressing nonphysical negativity issues and facilitates the representation of a mechanistic biogeochemical description in Earth system models to reduce climate prediction uncertainty.

show abstract

“…B6), which is nonphysical. One approach to avoid negative concentration is to scale back the update with a scaling factor (λ) (Bethke, 2007;Hammond, 2003) such that…”

Section: Scaling Clipping and Log Transformation For Avoiding Negatmentioning

confidence: 99%

Section: Scaling Clipping and Log Transformation For Avoiding Negatmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Addressing numerical challenges in introducing a reactive transport code into a land surface model: a biogeochemical modeling proof-of-concept with CLM–PFLOTRAN 1.0

et al. 2016

View full text Add to dashboard Cite

show abstract

“…There exist a number of studies dealing with the detailed strong and weak scaling behavior of various simulation platforms in hydrology and reactive solute transport, such as Hammond et al (2014), Kollet et al (2010), and Mills et al (2007). In these studies the focus has been placed on the parallel efficiency of solution algorithms including preconditioners for various classes and systems of partial differential equations in global implicit and explicit solution approaches.…”

Section: Introductionmentioning

confidence: 99%

Implementation and scaling of the fully coupled Terrestrial Systems Modeling Platform (TerrSysMP v1.0) in a massively parallel supercomputing environment – a case study on JUQUEEN (IBM Blue Gene/Q)

Gasper¹,

Goergen

Shrestha

et al. 2014

Geosci. Model Dev.

View full text Add to dashboard Cite

Abstract. Continental-scale hyper-resolution simulations constitute a grand challenge in characterizing nonlinear feedbacks of states and fluxes of the coupled water, energy, and biogeochemical cycles of terrestrial systems. Tackling this challenge requires advanced coupling and supercomputing technologies for earth system models that are discussed in this study, utilizing the example of the implementation of the newly developed Terrestrial Systems Modeling Platform (TerrSysMP v1.0) on JUQUEEN (IBM Blue Gene/Q) of the Jülich Supercomputing Centre, Germany. The applied coupling strategies rely on the Multiple Program Multiple Data (MPMD) paradigm using the OASIS suite of external couplers, and require memory and load balancing considerations in the exchange of the coupling fields between different component models and the allocation of computational resources, respectively. Using the advanced profiling and tracing tool Scalasca to determine an optimum load balancing leads to a 19 % speedup. In massively parallel supercomputer environments, the coupler OASIS-MCT is recommended, which resolves memory limitations that may be significant in case of very large computational domains and exchange fields as they occur in these specific test cases and in many applications in terrestrial research. However, model I/O and initialization in the petascale range still require major attention, as they constitute true big data challenges in light of future exascale computing resources. Based on a factor-two speedup due to compiler optimizations, a refactored coupling interface using OASIS-MCT and an optimum load balancing, the problem size in a weak scaling study can be increased by a factor of 64 from 512 to 32 768 processes while maintaining parallel efficiencies above 80 % for the component models.

show abstract

On the formation of multiple local peaks in breakthrough curves

Siirila‐Woodburn

Sánchez-Vila

Fernàndez-García

2015

Water Resources Research

View full text Add to dashboard Cite

The analysis of breakthrough curves (BTCs) is of interest in hydrogeology as a way to parameterize and explain processes related to anomalous transport. Classical BTCs assume the presence of a single peak in the curve, where the location and size of the peak and the slope of the receding limb has been of particular interest. As more information is incorporated into BTCs (for example, with high-frequency data collection, supercomputing efforts), it is likely that classical definitions of BTC shapes will no longer be adequate descriptors for contaminant transport problems. We contend that individual BTCs may display multiple local peaks depending on the hydrogeologic conditions and the solute travel distance. In such cases, classical definitions should be reconsidered. In this work, the presence of local peaks in BTCs is quantified from high-resolution numerical simulations in synthetic fields with a particle tracking technique and a kernel density estimator to avoid either overly jagged or smoothed curves that could mask the results. Individual BTCs from three-dimensional heterogeneous hydraulic conductivity fields with varying combinations of statistical anisotropy, heterogeneity models, and local dispersivity are assessed as a function of travel distance. The number of local peaks, their corresponding slopes, and a transport connectivity index are shown to strongly depend on statistical anisotropy and travel distance. Results show that the choice of heterogeneity model also affects the frequency of local peaks, but the slope is less sensitive to model selection. We also discuss how solute shearing and rerouting can be determined from local peak quantification.

show abstract

Evaluating the performance of parallel subsurface simulators: An illustrative example with PFLOTRAN

Cited by 289 publications

References 48 publications

Addressing numerical challenges in introducing a reactive transport code into a land surface model: a biogeochemical modeling proof-of-concept with CLM–PFLOTRAN 1.0

Addressing numerical challenges in introducing a reactive transport code into a land surface model: a biogeochemical modeling proof-of-concept with CLM–PFLOTRAN 1.0

Implementation and scaling of the fully coupled Terrestrial Systems Modeling Platform (TerrSysMP v1.0) in a massively parallel supercomputing environment – a case study on JUQUEEN (IBM Blue Gene/Q)

On the formation of multiple local peaks in breakthrough curves

Contact Info

Product

Resources

About