Model Evaluation and Performance

Abstract

Whether the behavior of a model matches the behavior of the (real) system–sufficiently well–has always been a matter of great interest. In lay terms, these are the essential questions one would like to have answered in seeking to evaluate a model: (i) has the model been constructed of approved materials, i.e. approved constituent hypotheses (in scientific terms)? (ii) does its behavior approximate well that observed in respect of the real thing? (iii) does it work, i.e. does it fulfill its designated task, or serve its intended purpose?

Characterising performance of environmental models

“…Studies have also focused explicitly on performance criteria, such as Moriasi et al (2007) and Dawson et al (2007Dawson et al ( , 2010, who produced guidelines for systematic model evaluation, including a list of recommended evaluation techniques and performance metrics. Beck (2006) provides a survey of key issues related to performance evaluation. And Matott et al (2009) reviewed model evaluation concepts in the context of integrated environmental models and discussed several relevant software-based tools.…”

Approaches to evaluating model quality across different regime types in environmental and public health governance

“…These include (Van der Sluijs et al, 2008;Oreskes et al, 1994;Beck, 2002;EPA, 2003;Rykiel, 1997;Clark and Majone, 1985;Refsgaard et al, 2007;Maxim and Van der Sluijs, 2011;Augusiak et al, 2014;Grimm et al, 2014): the acceptability of the normative commitments embodied in certain models (e.g. the implications of discount rates for intergenerational equity); the value judgments reflected -or omitted -in model structures and practices (e.g.…”

Uncertainty, assumptions and value commitments in the knowledge base of complex environmental problems

“…To give some examples, Hornberger andSpear (1981, cited in Saltelli, 2002) Beven's (2002) concept of equifinality (the phenomenon that models may be non-unique in their accuracy of both reproduction observations and prediction) and Beck's (2002) closely related notion that almost all models suffer from a lack of identifiability (many combinations of values for the model's parameters may permit the model to fit the observed data more or less equally well) further emphasise the problematic nature of models and model predictions as a source of knowledge for decision making. Yearley (1996) argued that values and value-laden assumptions enter into the formulation of environmental issues before the 'facts' are even established by science.…”