Sensitivity analysis of a sensitivity analysis: We are likely overlooking the impact of distributional assumptions

Paleari, Livia; Confalonieri, Roberto

doi:10.1016/j.ecolmodel.2016.09.008

Cited by 37 publications

(38 citation statements)

References 51 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…[], Marangoni et al . [], and Paleari and Confalonieri [] show the growing trend of sensitivity analysis investigations in the climate and environmental modeling arena, where also the need for keeping computational burden under control is felt. In this respect, the ensemble approach developed here might result in supporting investigators in such sectors as well.…”

Section: Discussionmentioning

confidence: 99%

Making the most out of a hydrological model data set: Sensitivity analyses to open the model black‐box

Borgonovo

Plischke

et al. 2017

Water Resources Research

View full text Add to dashboard Cite

In this work, we investigate methods for gaining greater insight from hydrological model runs conducted for uncertainty quantification and model differentiation. We frame the sensitivity analysis questions in terms of the main purposes of sensitivity analysis: parameter prioritization, trend identification, and interaction quantification. For parameter prioritization, we consider variance‐based sensitivity measures, sensitivity indices based on the L1‐norm, the Kuiper metric, and the sensitivity indices of the DELSA methods. For trend identification, we investigate insights derived from graphing the one‐way ANOVA sensitivity functions, the recently introduced CUSUNORO plots, and derivative scatterplots. For interaction quantification, we consider information delivered by variance‐based sensitivity indices. We rely on the so‐called given‐data principle, in which results from a set of model runs are used to perform a defined set of analyses. One avoids using specific designs for each insight, thus controlling the computational burden. The methodology is applied to a hydrological model of a river in Belgium simulated using the well‐established Framework for Understanding Structural Errors (FUSE) on five alternative configurations. The findings show that the integration of the chosen methods provides insights unavailable in most other analyses.

show abstract

Section: Discussionmentioning

confidence: 99%

Making the most out of a hydrological model data set: Sensitivity analyses to open the model black‐box

Borgonovo

Plischke

et al. 2017

Water Resources Research

View full text Add to dashboard Cite

show abstract

“…A third example is illustrated in [48], and is the case in which alternative scientific studies assign different distributions to a given model input. A fourth case is discussed in the works of [6], [4] and most recently [57], where a robustness question is asked directly by the analyst, who wishes to explore the stability of sensitivity analysis results for perturbations in the model input distributions.…”

Section: Interpretation and Methodological Aspects Concerning Aggregamentioning

confidence: 99%

“…Gao et al show high variability in computer experiments performed under alternative scenarios [26]. Paleari and Confalonieri test robustness of sensitivity results for uncertainty in distribution using the WARM model as a case study [57]. They show that uncertainty in distribution causes an overturn of the most important variables in 22% of the cases.…”

Section: Introductionmentioning

confidence: 99%

Functional ANOVA with Multiple Distributions: Implications for the Sensitivity Analysis of Computer Experiments

Borgonovo¹,

Morris²,

Plischke³

2018

SIAM/ASA J. Uncertainty Quantification

View full text Add to dashboard Cite

The functional ANOVA expansion of a multivariate mapping plays a fundamental role in statistics. The expansion is unique once a unique distribution is assigned to the covariates. Recent investigations in the environmental and climate sciences show that analysts may not be in a position to assign a unique distribution in realistic applications. We offer a systematic investigation of existence, uniqueness, orthogonality, monotonicity and ultramodularity of the functional ANOVA expansion of a multivariate mapping when a multiplicity of distributions is assigned to the covariates. In particular, we show that a multivariate mapping can be associated with a core of probability measures that guarantee uniqueness. We obtain new results for variance decomposition and dimension distribution under mixtures. Implications for the global sensitivity analysis of computer experiments are also discussed.of Beckman and McKay is possibly the first work discussing the stability of sensitivity analysis results for perturbations in the model input distributions [6]. As Saltelli et al. underline, the use of multiple distributions may controversial [73]. Nonetheless, it has become a de-facto part of several studies and is frequently adopted.Our purpose is to offer a systematic investigation of the impact of removing the unique distribution assumption on the classical functional ANOVA expansion of a multivariate mapping. We consider two paths that emerge from current and past practices. The starting datum is that the analyst posits a set M = µ 1In the first path, the analyst evaluates the model for each distribution in M separately and obtains sensitivity measures for each distribution -without-prior path, henceforth. In the second path, the analyst assigns a prior over the the distributions in M -with-prior path henceforth. For each path, we investigate six relevant notions: existence, uniqueness, orthogonality, monotonicity, ultramodularity, variance decomposition and dimension distribution. We study the implications in light of three sensitivity analysis settings: factor prioritization, trend identification and interaction quantification.Let us report some of the findings. In both paths, existence is ensured if all the posited measures are compatible with the functional ANOVA expansion of the input output mapping. Regarding uniqueness, in the without-prior path the analyst is dealing with as many functional ANOVA expansions as many are the cores in M. A core is defined as a set of probability measures that lead to identical expansions. Thus, one has uniqueness if all the posited measures belong to the same core. In the with-prior path, one regains uniqueness: a multivariate mapping can be uniquely represented as the mixture of functional ANOVA expansions. Regarding orthogonality, in the without-prior path it is preserved. In the with-prior path, mixtures of classical functional ANOVA effects are not orthogonal with respect to the mixture of the distributions in M. Regarding monotonicity and ultramodularity, in the without-prior path t...

show abstract

“…The robustness of Sobol indices to changes in the distributional uncertainty has been a focal point of several recent studies [122,55,56]. This may have important consequences since it has been shown that changes in the marginal distributions may change the ordering of the Sobol indices in [28].…”

Section: Robustness Of Sa Estimatesmentioning

confidence: 99%

Sensitivity analysis methods in the biomedical sciences

Qian

Mahdi

2020

Mathematical Biosciences

132

View full text Add to dashboard Cite

Sensitivity analysis is an important part of a mathematical modeller's toolbox for model analysis. In this review paper, we describe the most frequently used sensitivity techniques, discussing their advantages and limitations, before applying each method to a simple model. Also included is a summary of current software packages, as well as a modeller's guide for carrying out sensitivity analyses. Finally, we apply the popular Morris and Sobol methods to two models with biomedical applications, with the intention of providing a deeper understanding behind both the principles of these methods and the presentation of their results.

show abstract

Sensitivity analysis of a sensitivity analysis: We are likely overlooking the impact of distributional assumptions

Cited by 37 publications

References 51 publications

Making the most out of a hydrological model data set: Sensitivity analyses to open the model black‐box

Making the most out of a hydrological model data set: Sensitivity analyses to open the model black‐box

Functional ANOVA with Multiple Distributions: Implications for the Sensitivity Analysis of Computer Experiments

Sensitivity analysis methods in the biomedical sciences

Contact Info

Product

Resources

About