Framework for Quantification and Risk Analysis for Layered Uncertainty using Optimization: NASA UQ Challenge

Chaudhuri, Anirban; Waycaster, Garrett; Matsumura, Taiki; Price, Nathaniel B.; Haftka, Raphael T.

doi:10.2514/6.2014-1498

Cited by 5 publications

(5 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Based on these findings, we would choose to acquire reduced uncertainty models for parameters 5, 4, 18, and 20. However, we have already requested updated models during the first round of the NASA challenge problem [15] using surrogate methods we later found to be inaccurate. Because of this, we actually have reduced uncertainty models for parameters 21, 1, 18, and 16.…”

Section: Sensitivity Of Subparameters To Quantities Of Interest mentioning

confidence: 98%

“…A modified Kolmogorov-Smirnov (K-S) statistic, used here for comparing two ECDFs with n (given observations) and n 0 (randomly generated using aleatory uncertainty; 100 in this work) samples (D n;n 0 ), is given by Eq. (15). It is the sum of distances at the observed values between the two ECDFs.…”

Section: A Cdf Matching Uncertainty Quantification Approachmentioning

confidence: 99%

“…We observe from the first round of results for the NASA challenge problem that many participants, including ourselves, elected to use surrogates to replace some of the Table 7 Optimal values of fixed parameters corresponding to percent changes in Table 6 x 1 provided challenge problem functions. However, the types of surrogates, number of training points, and the quantities used in the approximation varied greatly [15][16][17][18][19][20][21]. It is therefore of interest to better understand how well some common surrogates perform for this problem, particularly since the challenge problem functions have several difficult attributes for fitting surrogates, such as high nonlinearity and high dimensionality.…”

Section: Sensitivity Of Subparameters To Intermediate Variables (mentioning

confidence: 99%

“… 15. Mean, median, and maximum (max) absolute relative error in ERR prediction versus MCS at 100 test points for a) J 1 , and b) J 2 .…”

mentioning

confidence: 99%

See 3 more Smart Citations

NASA Uncertainty Quantification Challenge: An Optimization-Based Methodology and Validation

Chaudhuri

Waycaster

Price

et al. 2015

Journal of Aerospace Information Systems

Self Cite

View full text Add to dashboard Cite

The focus of this work is on uncertainty characterization, sensitivity analysis, uncertainty propagation, and extreme-case analysis. To deal with the computationally expensive and complex NASA problem, a simpler toy problem is devised to mimic the NASA problem for which the true results were known. The toy problem helped in thoroughly testing the current methods and their repeatability. For uncertainty characterization, a novel cumulative density function matching method is proposed, which gave similar results as a standard Markov chain-Monte Carlobased Bayesian approach. An efficient reliability reanalysis-based probability-box sensitivity analysis method is employed to identify the most sensitive parameters to the risk analysis metrics. Uncertainty propagation to find extreme values for the risk analysis metrics is done using a single-loop efficient reliability reanalysis-based method. A modified version of the efficient reliability reanalysis is proposed that uses self-normalizing weights and caps on the weights; this is referred to as a capped self-normalizing efficient reliability reanalysis. This method showed considerably better performance at estimating risk analysis metrics for this application as compared to the generic efficient reliability reanalysis. The use of efficient reliability reanalysis was dictated by the cost of the black-box functions provided by NASA.Nomenclature D = modified Kolmogorov-Smirnov statistic F = cumulative density function f = probability density function g = requirement metrics I = indicator function J 1 = expected value of worst-case requirement metric J 2 = probability of failure n = given number of observations n 0 = number of aleatory samples for generating an empirical cumulative density function p = uncertain random variables q = sampling probability density function for efficient reliability reanalysis w = worst-case requirement metric x = intermediate variables θ= subparameters (epistemic uncertainty) Θ = vector of subparameters

show abstract

Section: Sensitivity Of Subparameters To Quantities Of Interest mentioning

confidence: 98%

Section: A Cdf Matching Uncertainty Quantification Approachmentioning

confidence: 99%

Section: Sensitivity Of Subparameters To Intermediate Variables (mentioning

confidence: 99%

“… 15. Mean, median, and maximum (max) absolute relative error in ERR prediction versus MCS at 100 test points for a) J 1 , and b) J 2 .…”

mentioning

confidence: 99%

See 2 more Smart Citations

NASA Uncertainty Quantification Challenge: An Optimization-Based Methodology and Validation

Chaudhuri

Waycaster

Price

et al. 2015

Journal of Aerospace Information Systems

Self Cite

View full text Add to dashboard Cite

show abstract

“…The proposed methodology is illustrated using the NASA Langley Uncertainty Quantification Challenge (referred to NASA-LUQC, in the rest of the paper) problem presented in detail by Crespo et al 11 The proposed sequential approach for uncertainty refinement is starkly different from existing refinement approaches, [12][13][14][15][16] all of which simultaneously identify all four candidates of refinement; such an approach is referred as all-at-once uncertainty refinement in this paper.…”

Section: Introductionmentioning

confidence: 99%

Uncertainty Reduction using Bayesian Inference and Sensitivity Analysis: A Sequential Approach to the NASA Langley Uncertainty Quantification Challenge

Sankararaman

2016

18th AIAA Non-Deterministic Approaches Conference

View full text Add to dashboard Cite

This paper presents a computational framework for uncertainty characterization and propagation, and sensitivity analysis under the presence of aleatory and epistemic uncertainty, and develops a rigorous methodology for efficient refinement of epistemic uncertainty by identifying important epistemic variables that significantly affect the overall performance of an engineering system. The proposed methodology is illustrated using the NASA Langley Uncertainty Quantification Challenge (NASA-LUQC) problem that deals with uncertainty analysis of a generic transport model (GTM). First, Bayesian inference is used to infer subsystem-level epistemic quantities using the subsystem-level model and corresponding data. Second, tools of variance-based global sensitivity analysis are used to identify four important epistemic variables (this limitation specified in the NASA-LUQC is reflective of practical engineering situations where not all epistemic variables can be refined due to time/budget constraints) that significantly affect system-level performance. The most significant contribution of this paper is the development of the sequential refinement methodology, where epistemic variables for refinement are not identified all-at-once. Instead, only one variable is first identified, and then, Bayesian inference and global sensitivity calculations are repeated to identify the next important variable. This procedure is continued until all 4 variables are identified and the refinement in the system-level performance is computed. The advantages of the proposed sequential refinement methodology over the all-at-once uncertainty refinement approach are explained, and then applied to the NASA Langley Uncertainty Quantification Challenge problem.

show abstract

Sensitivity of Input Epistemic Uncertainty on Nondeterministic Performance Estimates Using Nondeterministic Simulations

Hale

Patil

Roy

2017

Journal of Verification, Validation and Uncertainty Quantification

View full text Add to dashboard Cite

This paper examines various sensitivity analysis methods which can be used to determine the relative importance of input epistemic uncertainties on the uncertainty quantified performance estimate. The results from such analyses would then indicate which input uncertainties would merit additional study. The following existing sensitivity analysis methods are examined and described: local sensitivity analysis by finite difference, scatter plot analysis, variance-based analysis, and p-box-based analysis. As none of these methods are ideally suited for analysis of dynamic systems with epistemic uncertainty, an alternate method is proposed. This method uses aspects of both local sensitivity analysis and p-box-based analysis to provide improved computational speed while removing dependence on the assumed nominal model parameters.

show abstract

Framework for Quantification and Risk Analysis for Layered Uncertainty using Optimization: NASA UQ Challenge

Cited by 5 publications

References 17 publications

NASA Uncertainty Quantification Challenge: An Optimization-Based Methodology and Validation

NASA Uncertainty Quantification Challenge: An Optimization-Based Methodology and Validation

Uncertainty Reduction using Bayesian Inference and Sensitivity Analysis: A Sequential Approach to the NASA Langley Uncertainty Quantification Challenge

Sensitivity of Input Epistemic Uncertainty on Nondeterministic Performance Estimates Using Nondeterministic Simulations

Contact Info

Product

Resources

About