Linking big models to big data: efficient ecosystem model calibration through Bayesian model emulation

Abstract

Abstract. Data-model integration plays a critical role in assessing and improving our capacity to predict ecosystem dynamics. Similarly, the ability to attach quantitative statements of uncertainty around model forecasts is crucial for model assessment and interpretation and for setting field research priorities. Bayesian methods provide a rigorous data assimilation framework for these applications, especially for problems with multiple data constraints. However, the Markov chain Monte Carlo (MCMC) techniques underlying most Bayesian calibration can be prohibitive for computationally demanding models and large datasets. We employ an alternative method, Bayesian model emulation of sufficient statistics, that can approximate the full joint posterior density, is more amenable to parallelization, and provides an estimate of parameter sensitivity. Analysis involved informative priors constructed from a meta-analysis of the primary literature and specification of both model and data uncertainties, and it introduced novel approaches to autocorrelation corrections on multiple data streams and emulating the sufficient statistics surface. We report the integration of this method within an ecological workflow management software, Predictive Ecosystem Analyzer (PEcAn), and its application and validation with two process-based terrestrial ecosystem models: SIPNET and ED2. In a test against a synthetic dataset, the emulator was able to retrieve the true parameter values. A comparison of the emulator approach to standard brute-force MCMC involving multiple data constraints showed that the emulator method was able to constrain the faster and simpler SIPNET model's parameters with comparable performance to the brute-force approach but reduced computation time by more than 2 orders of magnitude. The emulator was then applied to calibration of the ED2 model, whose complexity precludes standard (brute-force) Bayesian data assimilation techniques. Both models are constrained after assimilation of the observational data with the emulator method, reducing the uncertainty around their predictions. Performance metrics showed increased agreement between model predictions and data. Our study furthers efforts toward reducing model uncertainties, showing that the emulator method makes it possible to efficiently calibrate complex models.

A near-term iterative forecasting system successfully predicts reservoir hydrodynamics and partitions uncertainty in real time

“…A current limitation to implementing a Bayesian Monte-Carlo Markov Chain approach is the computation time to execute the GLM simulation within a daily iterative workflow. Future work that uses emulators of GLM may be able to speed computation and allow for more robust estimation of the joint distribution model of parameters that represent both prior knowledge and observed data [e.g., Fer et al 2018 ]. Finally, our implementation of FLARE at FCR used historical inflow data in the forecast.…”

Assessing the response of forest productivity to climate extremes in Switzerland using model–data fusion

“…Despite the fact that this method allows us to combine multiple data sources and types, most studies have focused on the local scale. Hence, an important step forward is now to use large and diverse datasets in combination with DVMs at the regional scale (Cailleret, Bircher, Hartig, Hülsmann, & Bugmann, 2019;Fer et al, 2018;Minunno, Peltoniemi, et al, 2019;Thomas et al, 2017;Van Oijen et al, 2013).…”

Characterizing Dryland Ecosystems Using Remote Sensing and Dynamic Global Vegetation Modeling

“…However, an understanding of DGVM capabilities and limitations in an ecosystem is necessary to capture uncertainties. An evaluation of a DGVM should include model parametrization, sensitivity analysis (SA), calibration, and evaluation (Fer et al, 2018;Keenan et al, 2013;Kuppel et al, 2012;Pandit et al, 2019;Post et al, 2017;Renwick et al, 2019;Santaren et al, 2007;Wang et al, 2001). There is an information gap regarding DGVM evaluation in drylands and more specifically in regions where the drivers in ecosystem processes may vary across an elevation gradient.…”

“…For example the SA for net primary production (NPP) may yield different results than for GPP (e.g. Dietze et al, 2014;Fer et al, 2018). Thus, differences in SA between studies is expected and one should consider the SA method, model structure, and target variables and ecosystems when evaluating any DGVM model performance.…”

“…In general, global calibration methods such as Covariance Matrix Adaptation Evolution Strategy (CMAES; Hansen et al, 2003), Differential Evolution (DE; Storn and Price, 1997) and Bayesian approaches (e.g. Dietze et al, 2014;Fer et al, 2018) are preferable if their computational cost can be afforded. It is likely that some of the underestimation of GPP can be explained by the limited number of years used for calibration and validation of EDv2.2 in this study (Table 3.1).…”