Evaluating prediction uncertainty in simulation models

McKay, Michael D.; Morrison, J.; Upton, Steve

doi:10.1016/s0010-4655(98)00155-6

Cited by 83 publications

(43 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Quantitative measures such as Fourier amplitude test (FAST; Cukier et al, 1973), McKay main (McKay-1) and two-way (McKay-2) interaction analyses (McKay et al, 1999), and Sobol sensitivity indices (Sobol, 1990(Sobol, , 2001) provide such quantitative assessment. All quantitative methods are variance-based methods that use an ANOVAlike decomposition (Fisher, 1925) to identify the subset of parameters that dominate performance simulation, but differ in the way performance is simulated.…”

Section: Descriptionmentioning

confidence: 99%

“…The performance variance is computed through random sampling of the parameter space, making the method nonparametric The subset that maximizes the ratio is considered the most sensitive subset. McKay et al (1999) extended the concept to account for two-way parameter interactions (McKay-2) and their effect on the performance variance, assuming the parameters to be uncorrelated.…”

Section: Descriptionmentioning

confidence: 99%

See 1 more Smart Citation

Parameter sensitivity analysis of a 1-D cold region lake model for land-surface schemes

Guerrero

Pernica

Wheater

et al. 2017

Hydrol. Earth Syst. Sci.

View full text Add to dashboard Cite

Abstract. Lakes might be sentinels of climate change, but the uncertainty in their main feedback to the atmosphereheat-exchange fluxes -is often not considered within climate models. Additionally, these fluxes are seldom measured, hindering critical evaluation of model output. Analysis of the Canadian Small Lake Model (CSLM), a one-dimensional integral lake model, was performed to assess its ability to reproduce diurnal and seasonal variations in heat fluxes and the sensitivity of simulated fluxes to changes in model parameters, i.e., turbulent transport parameters and the light extinc-A C++ open-source software package, Problem Solving environment for Uncertainty Analysis and Design Exploration (PSUADE), was used to perform sensitivity analysis (SA) and identify the parameters that dominate model behavior. The generalized likelihood uncertainty estimation (GLUE) was applied to quantify the fluxes' uncertainty, comparing daily-averaged eddy-covariance observations to the output of CSLM. Seven qualitative and two quantitative SA methods were tested, and the posterior likelihoods of the modeled parameters, obtained from the GLUE analysis, were used to determine the dominant parameters and the uncertainty in the modeled fluxes. Despite the ubiquity of the equifinality issue -different parameter-value combinations yielding equivalent results -the answer to the question was unequivocal: K d , a measure of how much light penetrates the lake, dominates sensible and latent heat fluxes, and the uncertainty in their estimates is strongly related to the accuracy with which K d is determined. This is important since accurate and continuous measurements of K d could reduce modeling uncertainty.

show abstract

Section: Descriptionmentioning

confidence: 99%

Section: Descriptionmentioning

confidence: 99%

Parameter sensitivity analysis of a 1-D cold region lake model for land-surface schemes

Guerrero

Pernica

Wheater

et al. 2017

Hydrol. Earth Syst. Sci.

View full text Add to dashboard Cite

show abstract

“…Thus, the number of coefficients to compute in the decomposition is M +1 = 3 (15). The multivariate polynomials Φ j are given by (16):…”

Section: Pc-based Sensitivity Indicesmentioning

confidence: 99%

“…However, some models may be complex with a high number of parameters so that analytical calculations of the sensitivity indices become time consuming or even impossible. It is therefore necessary to estimate them [3,13,14,15,7]. Very often, they are computed using Monte Carlo simulations, but for computationally demanding models, this can become intractable.…”

Section: Introductionmentioning

confidence: 99%

Sensitivity study of dynamic systems using polynomial chaos

Sandoval

Anstett-Collin

Basset

2012

Reliability Engineering & System Safety

View full text Add to dashboard Cite

Global sensitivity has mainly been analyzed in static models, though most physical systems can be described by differential equations. Very few approaches have been proposed for the sensitivity of dynamic models and the only ones are local. Nevertheless, it would be of great interest to consider the entire uncertainty range of parameters since they can vary within large intervals depending on their meaning. Other advantage of global analysis is that the sensitivity indices of a given parameter are evaluated while all the other parameters can be varied. In this way, the relative variability of each parameter is taken into account, revealing any possible interactions. This paper presents the global sensitivity analysis for dynamic models with an original approach based on the polynomial chaos (PC) expansion of the output. The evaluation of the PC expansion of the output is less expensive compared to direct simulations. Moreover, at each time instant, the coefficients of the PC decomposition convey the parameter sensitivity and then a sensitivity function can be obtained. The PC coefficients are determined using non-intrusive methods. The proposed approach is illustrated with some well-known dynamic systems.

show abstract

“…A popular category of the existing PSA techniques belongs to the so-called variance-based methods, including the Fourier Amplitude Sensitivity Test (FAST) 6,7 , correlation ratios 8 or importance measures 9 , and Sobol' indices 10,11 , etc. Variance-based methods are derived from the decomposition of the total variance of a model response to different variation sources and their combinations.…”

Section: Introductionmentioning

confidence: 99%

Probabilistic Sensitivity Analysis Methods for Design Under Uncertainty

Liu¹,

Chen²,

Sudjianto

2004

10th AIAA/ISSMO Multidisciplinary Analysis and Optimization Conference

View full text Add to dashboard Cite

Sensitivity analysis (SA) is an important procedure in engineering design to obtain valuable information about the model behavior to guide a design process. For design under uncertainty, probabilistic sensitivity analysis (PSA) methods have been developed to provide insight into the probabilistic behavior of a model. In this paper, the goals of PSA at different design stages are investigated. In the prior-design stage, PSA can be utilized to identify those probabilistically non-significant variables and reduce the dimension of a random design space. It can reduce the computational cost associated with uncertainty assessment without much sacrifice on the optimum solution. For post-design analysis, probabilistic sensitivity analysis can be used to identify where to spend design resources for the largest potential improvement of a performance. Based on the interested distribution range of a random response, the PSA methods can be categorized into two types: the global response probabilistic sensitivity analysis (GRPSA) and the regional response probabilistic sensitivity analysis (RRPSA). Four widely-used PSA methods: Sobol' indices, Wu's sensitivity coefficients, the MPP based sensitivity coefficients, and the Kullback-Leibler entropy based method are selected for comparison. The merits behind each method are reviewed in details. Their advantages, limitations, and applicability are investigated. Their effectiveness and applicability under different design scenarios are compared in two numerical examples and two engineering design problems.

show abstract

Evaluating prediction uncertainty in simulation models

Cited by 83 publications

References 14 publications

Parameter sensitivity analysis of a 1-D cold region lake model for land-surface schemes

Parameter sensitivity analysis of a 1-D cold region lake model for land-surface schemes

Sensitivity study of dynamic systems using polynomial chaos

Probabilistic Sensitivity Analysis Methods for Design Under Uncertainty

Contact Info

Product

Resources

About