Evaluating prediction uncertainty

McKay, Michael D.

doi:10.2172/29432

Cited by 97 publications

(61 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…McKay [4] proposed a more efficient estimation method based on the use of a single replicated Latin hypercube sampling (r-LHS) design for all K inputs. It should be noted that even with this efficiency improvement the main effect analysis is still very expensive requiring a substantial number (for example, thousands) of model evaluations.…”

Section: Main Effect Analysismentioning

confidence: 99%

“…This paper focuses on efficient and accurate methods for computing the first-and second-order sensitivity indices. Specifically, McKay's [4] main effect analysis is an efficient method for computing the first-order sensitivity indices. However, a difficulty when applying this method is the determination of a suitable sample size to achieve sufficient accuracy.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Self-validated variance-based methods for sensitivity analysis of model outputs

Tong

2010

Reliability Engineering & System Safety

View full text Add to dashboard Cite

Global sensitivity analysis (GSA) has the advantage over local sensitivity analysis in that GSA does not require strong model assumptions such as linearity or monotonicity. As a result, GSA methods such as those based on variance decomposition are well-suited to multi-physics models, which are often plagued by large nonlinearities. However, as with many other sampling-based methods, inadequate sample size can badly pollute the result accuracies. A natural remedy is to adaptively increase the sample size until sufficient accuracy is obtained. This paper proposes an iterative methodology comprising mechanisms for guiding sample size selection and self-assessing result accuracy. The elegant features in the the proposed methodology are the adaptive refinement strategies for stratified designs. We first apply this iterative methodology to the design of a self-validated first-order sensitivity analysis algorithm. We also extend this methodology to design a self-validated second-order sensitivity analysis algorithm based on refining replicated orthogonal array designs. Several numerical experiments are given to demonstrate the effectiveness of these methods.

show abstract

Section: Main Effect Analysismentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Self-validated variance-based methods for sensitivity analysis of model outputs

Tong

2010

Reliability Engineering & System Safety

View full text Add to dashboard Cite

show abstract

“…Secondly, it can cope with group of random inputs which is the key of our methodology to estimate the influence of dynamic inputs. Note that, although Quasi Monte Carlo sampling technique is known to have a better coverage of the input space [14,23], LHS is preferred here because we have to deal with a very high number of inputs (thousands) and the sampling design proposed in [22] (similar to rLHS [24]) is suited to LHS. Then, a second input LH sample Ω 2 is generated from Ω 1 by arbitrarily defining an independent rank matrix R 2 of size N s ⇥ d and setting,…”

Section: Estimating the Sensitivity Indices With Two Lhs Samplesmentioning

confidence: 99%

“…Note that, the new Ω 2 is also an LH sample and this column-wise permutation trick is equivalent to the replicated LH sampling [24]. After running the model with this new sample, a second model response vector…”

Section: Estimating the Sensitivity Indices With Two Lhs Samplesmentioning

confidence: 99%

“…However, the Pick and Freeze method is computationally demanding because it requires N ⇥(N d +1) model runs where N d is the number of input factors and N is the sample size. In our study, we employ the random balance design trick (also used in replicated Latin Hypercube [24]) that allows to estimate the entire set of first-order sensitivity indices with only two samples of size N (see [22]). …”

Section: Consider Thementioning

confidence: 99%

See 1 more Smart Citation

Sensitivity analysis of complex models: Coping with dynamic and static inputs

Anstett-Collin

Goffart

Mara³

et al. 2015

Reliability Engineering & System Safety

View full text Add to dashboard Cite

. Sensitivity analysis of complex models: coping with dynamic and static inputs. Reliability Engineering and System Safety, Elsevier, 2015, 134, pp.268-275 AbstractIn this article, we address the issue of conducting a sensitivity analysis of complex models with both static and dynamic uncertain inputs. While several approaches have been proposed to compute the sensitivity indices of the static inputs (i.e. parameters), the ones of the dynamic inputs (i.e. stochastic fields) have been rarely addressed. For this purpose, we first treat each dynamic as a Gaussian process. Then, the truncated Karhunen-Loève expansion of each dynamic input is performed. Such an expansion allows to generate independent Gaussian processes from a finite number of independent random variables. Given that a dynamic input is represented by a finite number of random variables, its variance-based sensitivity index is defined by the sensitivity index of this group of variables. Besides, an efficient samplingbased strategy is described to estimate the first-order indices of all the input factors by only using two input samples. The approach is applied to a buildPreprint submitted to Reliability Engineering and System Safety August 5, 2014ing energy model, in order to assess the impact of the uncertainties of the material properties (static inputs) and the weather data (dynamic inputs) on the energy performance of a real low energy consumption house.

show abstract

A role for sensitivity analysis in presenting the results from MCDA studies to decision makers

Saltelli

Tarantola

Chan

1999

J. Multi-Crit. Decis. Anal.

View full text Add to dashboard Cite

The aim of sensitivity analysis (SA) is to ascertain how much the uncertainty in the output of a model is influenced by the uncertainty in its input factors. An SA can be performed using different methods, which are classified according to various criteria. One possible classification is that in which global and local approaches are identified. This paper strengthens the role of global SA methods and suggests their use in the context of MCDA. Useful applications of global SA already exist in a variety of fields where numerical models are considered, e.g. in economics, engineering and chemistry. Global SA is quite different in its formulation and application from local SA, which is seen more frequently in the literature. The global sensitivity indices adopted are derived from a variance decomposition scheme: they can be estimated by alternative computational strategies, such as the extended FAST or the method proposed by Sobol'. A truly global SA is capable of apportioning the output uncertainty according to any subgroup of input factors. Hence, the output uncertainty of an MCDA can, for instance, be decomposed into a part due to uncertain model inputs and a part due to poorly defined (or variable) weights attached to the criteria. This information could be useful to the decision maker (DM) since it explains synthetically how much the assessment of an MCDA study is biased by the assessor judgements. Alternative regrouping of the uncertain input elements might shed light on other features of the problem addressed by the DM.

show abstract

Evaluating prediction uncertainty

Cited by 97 publications

References 25 publications

Self-validated variance-based methods for sensitivity analysis of model outputs

Self-validated variance-based methods for sensitivity analysis of model outputs

Sensitivity analysis of complex models: Coping with dynamic and static inputs

A role for sensitivity analysis in presenting the results from MCDA studies to decision makers

Contact Info

Product

Resources

About