Global sensitivity analysis for complex ecological models: a case study of riparian cottonwood population dynamics

Abstract

Mechanism-based ecological models are a valuable tool for understanding the drivers of complex ecological systems and for making informed resource-management decisions. However, inaccurate conclusions can be drawn from models with a large degree of uncertainty around multiple parameter estimates if uncertainty is ignored. This is especially true in nonlinear systems with multiple interacting variables. We addressed these issues for a mechanism-based, demographic model of Populus fremontii (Fremont cottonwood), the dominant riparian tree species along southwestern U.S. rivers. Many cottonwood populations have declined following widespread floodplain conversion and flow regulation. As a result, accurate predictive models are needed to analyze effects of future climate change and water management decisions. To quantify effects of parameter uncertainty, we developed an analytical approach that combines global sensitivity analysis (GSA) with classification and regression trees (CART) and Random Forest, a bootstrapping CART method. We used GSA to quantify the interacting effects of the full range of uncertainty around all parameter estimates, Random Forest to rank parameters according to their total effect on model predictions, and CART to identify higher-order interactions. GSA simulations yielded a wide range of predictions, including annual germination frequency of 10-100%, annual first-year survival frequency of 0-50%, and patch occupancy of 0-100%. This variance was explained primarily by complex interactions among abiotic parameters including capillary fringe height, stage-discharge relationship, and floodplain accretion rate, which interacted with biotic factors to affect survival. Model precision was primarily influenced by well-studied parameter estimates with minimal associated uncertainty and was virtually unaffected by parameter estimates for which there are no available empirical data and thus a large degree of uncertainty. Therefore, research to improve model predictions should not always focus on the least-studied parameters, but rather those to which model predictions are most sensitive. We advocate the combined use of global sensitivity analysis, CART, and Random Forest to: (1) prioritize research efforts by ranking variable importance; (2) efficiently improve models by focusing on the most important parameters; and (3) illuminate complex model properties including nonlinear interactions. We present an analytical framework that can be applied to any model with multiple uncertain parameter estimates.

Sensitivity Analysis of Environmental Models: A Systematic Review with Practical Workflow

“…A particularly interesting class of nonlinear regression methods in the context of Sensitivity Analysis is that of Classification And Regression Trees (CART, for application examples see e.g. Harper et al (2011) and Singh et al (2014)). CART provides several advantages, including that they can easily handle non-numerical inputs and outputs, and that they can be used for both ranking and mapping.…”

“…Global SA is used for a range of very diverse purposes, including: to support model calibration, verification, diagnostic evaluation or simplification (e.g. Sieber and Uhlenbrook, 2005;Harper et al, 2011;Nossent et al, 2011;Kelleher et al, 2013;Shin et al, 2013;Butler et al, 2014); to prioritize efforts for uncertainty reduction (e.g. Hamm et al, 2006); to analyse the dominant controls of a system (e.g.…”

Quantifying model uncertainty to improve watershed-level ecosystem service quantification: a global sensitivity analysis of the RUSLE

“…Model accuracy assessments are particularly challenging in data-poor regions and developing countries. However, a sensitivity analysis can efficiently provide data to reduce uncertainty in model predictions and decision-making (Harper et al 2011). Sensitivity analyses assess both the parameters' influence on prediction uncertainty and the influence of parameter estimation (input data) on uncertainty (Saltelli et al 1999;Harper et al 2011).…”

“…LSA estimates the contribution of each model factor (or parameter, hereafter just mention factor) to model predictions by varying each model factor singly while holding other factors constant (Saltelli et al 1999). LSA is a constructive analysis, but it does not capture interactions among factors and interaction effects on model predictions (Wagner 1995;Harper et al 2011). Global sensitivity analysis (GSA) is considered a more robust approach because it considers higher order interactions among factors to assess model behavior and to estimate factor importance (Harper et al 2011).…”

Measuring regional resilience towards fossil fuel supply constraints. Adaptability and vulnerability in socio-ecological Transformations-the case of Austria

“…It thus allows to examine the contribution and behavior of each independent variable (Genuer et al, 2010). Hence, for this application, the variable "importance" can be understood as the sensitivity of the continuous variable "resilience" to the different predictors (Harper et al, 2011). Moreover, random forest calculates a measure of the models accuracy called "Out Of Bag" (OOB).…”