Application of a Versatile "Real-Space" Validation Methodology to a Fire Model

Romero, Vicente J.; Luketa, Anay; Sherman, M.P.

doi:10.2514/1.46358

Cited by 11 publications

(10 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…These uncertainties are defined prior to the current experiment (they are not determined by or in the experiment) and come to the experiment model as a priori uncertainties in model form and/or parameter values. See [28], [40], [46], [49], [51], [52], and [56] for illustration of how the framework handles these traveling "model-intrinsic" uncertainties in model validation and conditioning.…”

Section: "Experiments Model" and "Traveling Model"mentioning

confidence: 99%

“…This is due to the way the framework uses the relative sizes of the experimental and simulation uncertainty bars to gauge model adequacy. See [47], [49], [52] for further explanation and examples.…”

Section: Deciding On the Formulation Or Metric Of Discrepancy Charactmentioning

confidence: 99%

“…Uncertainties in the non-traveling aspects depend on the design and implementation of the experiments; the diagnostic instrumentation used; and the chosen scope of the e-model, all of which should be optimized to reduce the non-traveling uncertainty as much as achievable within project constraints (see section 5 for elaboration). It is also important to note that uncertainties in the experiments are treated differently in the Real Space framework if the uncertainties are affiliated with traveling aspects versus non-traveling aspects ( [49], [51], [52]). …”

Section: "Experiments Model" and "Traveling Model"mentioning

confidence: 99%

“…The affiliated concept of Type X error in model validation or conditioning is discussed. Data conditioning procedures ( [47]- [49]) that can effectively mitigate prediction risk associated with Type X error are also discussed. Connections are drawn between these new concepts to the literature and prior concepts of Type I and II errors and Model Builder's risk and Model User's risk.…”

Section: Introductionmentioning

confidence: 99%

“…• device thermal response and initiated failure [43], [46] • device thermal-structural response and failure [51] • foam thermal pyrolysis and vaporization [28], [33], [53] • fire and object-response modeling [40], [49] • radiation-damaged electronic device response [56].…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Elements of a pragmatic approach for dealing with bias and uncertainty in experiments through predictions : experiment design and data conditioning; %22real space%22 model validation and conditioning; hierarchical modeling and extrapolative prediction.

Romero¹

2011

Self Cite

View full text Add to dashboard Cite

This report explores some important considerations in devising a practical and consistent framework and methodology for utilizing experiments and experimental data to support modeling and prediction. A pragmatic and versatile "Real Space" approach is outlined for confronting experimental and modeling bias and uncertainty to mitigate risk in modeling and prediction. The elements of experiment design and data analysis, data conditioning, model conditioning, model validation, hierarchical modeling, and extrapolative prediction under uncertainty are examined. An appreciation can be gained for the constraints and difficulties at play in devising a viable end-to-end methodology. Rationale is given for the various choices underlying the Real Space end-to-end approach. The approach adopts and refines some elements and constructs from the literature and adds pivotal new elements and constructs. Crucially, the approach reflects a pragmatism and versatility derived from working many industrial-scale problems involving complex physics and constitutive models, steady-state and time-varying nonlinear behavior and boundary conditions, and various types of uncertainty in experiments and models. The framework benefits from a broad exposure to integrated experimental and modeling activities in the areas of heat transfer, solid and structural mechanics, irradiated electronics, and combustion in fluids and solids.4

show abstract

Section: "Experiments Model" and "Traveling Model"mentioning

confidence: 99%

Section: Deciding On the Formulation Or Metric Of Discrepancy Charactmentioning

confidence: 99%

Section: "Experiments Model" and "Traveling Model"mentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Elements of a pragmatic approach for dealing with bias and uncertainty in experiments through predictions : experiment design and data conditioning; %22real space%22 model validation and conditioning; hierarchical modeling and extrapolative prediction.

Romero¹

2011

Self Cite

View full text Add to dashboard Cite

show abstract

Separation of aleatory and epistemic uncertainty in probabilistic model validation

Mullins

You

Mahadevan

et al. 2016

Reliability Engineering & System Safety

View full text Add to dashboard Cite

This paper investigates model validation under a variety of different data scenarios and clarifies how different validation metrics may be appropriate for different scenarios. In the presence of multiple uncertainty sources, model validation metrics that compare the distributions of model prediction and observation are considered. Both ensemble validation and point-by-point approaches are discussed, and it is shown how applying the model reliability metric point-by-point enables the separation of contributions from aleatory and epistemic uncertainty sources. After individual validation assessments are made at different input conditions, it may be desirable to obtain an overall measure of model validity across the entire domain. This paper proposes an integration approach that assigns weights to the validation results according to the relevance of each validation test condition to the overall intended use of the model in prediction. Since uncertainty propagation for probabilistic validation is often unaffordable for complex computational models, surrogate models are often used; this paper proposes an approach to account for the additional uncertainty introduced in validation by the uncertain fit of the surrogate model. The proposed methods are demonstrated with a microelectromechanical system (MEMS) example.

show abstract

Toward a Better Understanding of Model Validation Metrics

Liu

Chen

Arendt

et al. 2011

Journal of Mechanical Design

139

View full text Add to dashboard Cite

Model validation metrics have been developed to provide a quantitative measure that characterizes the agreement between predictions and observations. In engineering design, the metrics become useful for model selection when alternative models are being considered. Additionally, the predictive capability of a computational model needs to be assessed before it is used in engineering analysis and design. Due to the various sources of uncertainties in both computer simulations and physical experiments, model validation must be conducted based on stochastic characteristics. Currently there is no unified validation metric that is widely accepted. In this paper, we present a classification of validation metrics based on their key characteristics along with a discussion of the desired features. Focusing on stochastic validation with the consideration of uncertainty in both predictions and physical experiments, four main types of metrics, namely classical hypothesis testing, Bayes factor, frequentist’s metric, and area metric, are examined to provide a better understanding of the pros and cons of each. Using mathematical examples, a set of numerical studies are designed to answer various research questions and study how sensitive these metrics are with respect to the experimental data size, the uncertainty from measurement error, and the uncertainty in unknown model parameters. The insight gained from this work provides useful guidelines for choosing the appropriate validation metric in engineering applications.

show abstract

Application of a Versatile "Real-Space" Validation Methodology to a Fire Model

Cited by 11 publications

References 12 publications

Elements of a pragmatic approach for dealing with bias and uncertainty in experiments through predictions : experiment design and data conditioning; %22real space%22 model validation and conditioning; hierarchical modeling and extrapolative prediction.

Elements of a pragmatic approach for dealing with bias and uncertainty in experiments through predictions : experiment design and data conditioning; %22real space%22 model validation and conditioning; hierarchical modeling and extrapolative prediction.

Separation of aleatory and epistemic uncertainty in probabilistic model validation

Toward a Better Understanding of Model Validation Metrics

Contact Info

Product

Resources

About