A stochastic approach to model validation

Luis, Steven J.; McLaughlin, Dennis K.

doi:10.1016/0309-1708(92)90029-2

Cited by 58 publications

(78 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The errors are assessed using the goodness of fit criteria which compare observations with predictions (Luis and McLaughlin, 1992;Fawcett et al, 1995). The source of the observed data used to validate models can be either be analytical solutions (Horritt, 2000), laboratory scale models (Thomas and Williams, 1995), field data (Lane et al, 1999) or remotely sensed data (Horritt, 2000).…”

Section: General Modelling Proceduresmentioning

confidence: 99%

“…The availability of data is one of the main controls on what process representations are chosen for models, while the data quality influences the quality of the model results. Luis and McLaughlin (1992) note the importance of the data which is used to assess models with, as well as the data used within the model. The validation dataset has inherent measurement errors associated with it.…”

Section: Review Of Hydrological Modelsmentioning

confidence: 99%

See 1 more Smart Citation

Rural land management impacts on catchment scale flood risk

Pattison¹

2010

Role of Hydrology in Managing Consequences of a Changing Global Environment

View full text Add to dashboard Cite

The full-text may be used and/or reproduced, and given to third parties in any format or medium, without prior permission or charge, for personal research or study, educational, or not-for-pro t purposes provided that:• a full bibliographic reference is made to the original source• a link is made to the metadata record in Durham E-Theses• the full-text is not changed in any way The full-text must not be sold in any format or medium without the formal permission of the copyright holders.Please consult the full Durham E-Theses policy for further details. Most approaches to this problem aim to upscale from individual grid cells to whole catchments, something that restricts the complexity of possible process representation, produces models that may not be parsimonious with the data needed to calibrate them and, faced with data uncertainties, provides computational limitations on the extent to which model uncertainty can be fully explored. Rather than upscaling to problems of concern, this thesis seeks to downscale from locations of known flood risk, as a means of identifying where land use management changes might be beneficial and then uses numerical modelling to identify the kinds of management changes required in those downscaled locations. Thus, the aim of this thesis is to test an approach to understanding the impacts of rural land management upon flood risk based upon catchment-to-source downscaling.This thesis uses the case study of the River Eden catchment (2400 km 2 ) as a test (1) it was shown to have a significant impact on downstream flood risk; and (2) it had range of data and information needed for modelling land use changes.The second part of this thesis explored the land management scenarios that could be used to reduce flood risk at the catchment scale. The scenarios to be tested were determined through a stakeholder participation approach, whereby workshops were held to brainstorm and prioritise land management options, and then to identify specific locations within the Eamont sub-catchment where they could tested. There were two main types of land management scenarios chosen: (1) landscape-scale changes, including afforestation and compaction; and (2) corresponding to a 5.8% decrease in peak discharge. A key conclusion is that land management practices have been shown to have an effect on catchment scale flooding, even for extreme flood events. However, the effect of land management scenarios are both spatially and temporally dependent i.e. the same land management practice has different effects depending on where it is implemented, and when implemented in the same location has different effects on different flood events.

show abstract

Section: General Modelling Proceduresmentioning

confidence: 99%

Section: Review Of Hydrological Modelsmentioning

confidence: 99%

Rural land management impacts on catchment scale flood risk

Pattison¹

2010

Role of Hydrology in Managing Consequences of a Changing Global Environment

View full text Add to dashboard Cite

show abstract

“…A method that has been proposed (see Luis and McLaughlin, 1992) to test for model validity is to plot confidence intervals for the uncertainty of the differences for each difference, and evaluate the number of measurements that are outside these intervals. If 5% of these measurements lie outside the interval, Luis and McLaughlin suggests that we reject the model as valid.…”

Section: Two Measurement Timesmentioning

confidence: 99%

Statistical Validation of Engineering and Scientific Models: A Maximum Likelihood Based Metric

Hills¹,

Trucano²

2002

View full text Add to dashboard Cite

Two major issues associated with model validation are addressed here. First, we present a maximum likelihood approach to define and evaluate a model validation metric. The advantage of this approach is it is more easily applied to nonlinear problems than the methods presented earlier by Trucano (1999, 2001); the method is based on optimization for which software packages are readily available; and the method can more easily be extended to handle measurement uncertainty and prediction uncertainty with different probability structures. Several examples are presented utilizing this metric. We show conditions under which this approach reduces to the approach developed previously by Hills and Trucano (2001).Secondly, we expand our earlier discussions Trucano, 1999, 2001) on the impact of multivariate correlation and the effect of this on model validation metrics. We show that ignoring correlation in multivariate data can lead to misleading results, such as rejecting a good model when sufficient evidence to do so is not available.

show abstract

“…Beven [2002] summarizes strong arguments that models and theories in the environmental sciences are nothing else but hypotheses. A prominent example is the work by Luis and McLaughlin [1992], who indeed approach model validation via formal statistical hypothesis testing. Consequently, modelers and scientists should admit the hypothesis-like character of models and their underlying theories, conceptualizations, assumptions, parameterizations, and parameter values.…”

Section: Introductionmentioning

confidence: 99%

A hypothesis‐driven approach to optimize field campaigns

Nowak¹,

Rubin²,

Barros³

2012

Water Resources Research

View full text Add to dashboard Cite

[1] Most field campaigns aim at helping in specified scientific or practical tasks, such as modeling, prediction, optimization, or management. Often these tasks involve binary decisions or seek answers to yes/no questions under uncertainty, e.g., Is a model adequate? Will contamination exceed a critical level? In this context, the information needs of hydro(geo)logical modeling should be satisfied with efficient and rational field campaigns, e.g., because budgets are limited. We propose a new framework to optimize field campaigns that defines the quest for defensible decisions as the ultimate goal. The key steps are to formulate yes/no questions under uncertainty as Bayesian hypothesis tests, and then use the expected failure probability of hypothesis testing as objective function. Our formalism is unique in that it optimizes field campaigns for maximum confidence in decisions on model choice, binary engineering or management decisions, or questions concerning compliance with environmental performance metrics. It is goal oriented, recognizing that different models, questions, or metrics deserve different treatment. We use a formal Bayesian scheme called PreDIA, which is free of linearization, and can handle arbitrary data types, scientific tasks, and sources of uncertainty (e.g., conceptual, physical, (geo)statistical, measurement errors). This reduces the bias due to possibly subjective assumptions prior to data collection and improves the chances of successful field campaigns even under conditions of model uncertainty. We illustrate our approach on two instructive examples from stochastic hydrogeology with increasing complexity.

show abstract

A stochastic approach to model validation

Cited by 58 publications

References 15 publications

Rural land management impacts on catchment scale flood risk

Rural land management impacts on catchment scale flood risk

Statistical Validation of Engineering and Scientific Models: A Maximum Likelihood Based Metric

A hypothesis‐driven approach to optimize field campaigns

Contact Info

Product

Resources

About