Ten iterative steps in development and evaluation of environmental models

Abstract

Models are increasingly being relied upon to inform and support natural resource management. They are incorporating an ever broader range of disciplines and now often confront people without strong quantitative or model-building backgrounds. These trends imply a need for wider awareness of what constitutes good model-development practice, including reporting of models to users and sceptical review of models by users. To this end the paper outlines ten basic steps of good, disciplined model practice. The aim is to develop purposeful, credible models from data and prior knowledge, in consort with end-users, with every stage open to critical review and revision. Best practice entails identifying clearly the clients and objectives of the modelling exercise; documenting the nature (quantity, quality, limitations) of the data used to construct and test the model; providing a strong rationale for the choice of model family and features (encompassing review of alternative approaches); justifying the techniques used to calibrate the model; serious analysis, testing and discussion of model performance; and making a resultant statement of model assumptions, utility, accuracy, limitations, and scope for improvement. In natural resource management applications, these steps will be a learning process, even a partnership, between model developers, clients and other interested parties.

CFD simulation of pedestrian-level wind conditions around buildings: Past achievements and prospects

“…Several of these BPG documents were established specifically for the assessment of pedestrian-level wind conditions (Franke et al 2004, Blocken et al 2007, Tominaga et al 2008. Other, more general guidelines, but also of relevance to pedestrian-level wind conditions, are those by Casey and Wintergerste (2000), Jakeman et al (2006), Franke et al (2007 and Blocken and Gualtieri (2012). While papers focusing on guidelines are very important, they are not included in the VSI when they do not report specific CFD simulations.…”

Deliberative processes for comprehensive evaluation of agroecological models. A review

“…Evolution in model evaluation approaches, also accompanied by the creation of dedicated software tools (Fila et al 2003a, b;Tedeschi 2006;Criscuolo et al 2007;Olesen and Chang 2010), has culminated in reviews and position papers (Bellocchi et al 2010;Alexandrov et al 2011;Bennett et al 2013) with the aim of characterizing the performance of models and providing standards for publishing models in forms suitable for use by broad communities (Jakeman et al 2006;Laniak et al 2013). Several evaluation methods are available, but, usually, only a limited number of methods are used in modeling projects (as documented, for instance, by Richter et al 2012 andRitter andMuñoz-Carpena 2013), often due to time and resource constraints.…”

“…As such, stakeholder engagement is a fundamental, and perhaps defining, aspect of model evaluation (step 6, orange box). The evaluation of model adequacy is an essential step of the modeling process, either to build confidence in a model or to select alternative models (Jakeman et al 2006). The concept of evaluation, in spite of controversial terminology (Konikow and Bredehoeft 1992;Bredehoeft and Konikow 1993;Bair 1994;Oreskes 1998), is quite generally interpreted in terms of model suitability for a particular purpose, which means that a model is valuable and sound if it accomplishes what is expected of it (Hamilton 1991;Landry and Oral 1993;Rykiel 1996;Sargent 2001) or helps achieve a successful outcome.…”

Making Progress in Integrated Modelling and Environmental Decision Support

“…Adopting good modelling practices improves the chances for the reusability and extendibility of models. The Position Paper [4] on the development and evaluation of environmental models is just one example of a host of literature and recommended practice that includes Refsgaard [5], Van der Sluijs et al [6] and Lee [7].…”