Hydrologic and Water Quality Models: Performance Measures and Evaluation Criteria

Abstract

Performance measures (PMs) and corresponding performance evaluation criteria (PEC) are important aspects of calibrating and validating hydrologic and water quality models and should be updated with advances in modeling science. We synthesized PMs and PEC from a previous special collection, performed a meta-analysis of performance data reported in recent peer-reviewed literature for three widely published watershed-scale models (SWAT, HSPF, WARMF), and one field-scale model (ADAPT), and provided guidelines for model performance evaluation. Based on the synthesis, meta-analysis, and personal modeling experiences, we recommend coefficient of determination (R 2 ; in conjunction with gradient and intercept of the corresponding regression line), Nash Sutcliffe efficiency (NSE), index of agreement (d), root mean square error (RMSE; alongside the ratio of RMSE and standard deviation of measured data, RSR), percent bias (PBIAS), and several graphical PMs to evaluate model performance. We recommend that model performance can be judged "satisfactory" for flow simulations if monthly R 2 > 0.70 and d > 0.75 for field-scale models, and daily, monthly, or annual R 2 > 0.60, NSE > 0.50, and PBIAS ≤ ±15% for watershed-scale models. Model performance at the watershed scale can be evaluated as "satisfactory" if monthly R 2 > 0.40 and NSE > 0.45 and daily, monthly, or annual PBIAS ≤ ±20% for sediment; monthly R 2 > 0.40 and NSE > 0.35 and daily, monthly, or annual PBIAS ≤ ±30% for phosphorus (P); and monthly R 2 > 0.30 and NSE > 0.35 and daily, monthly, or annual PBIAS ≤ ±30% for nitrogen (N). For RSR, we recommend that previously published PEC be used as detailed in this article. We also recommend that these PEC be used primarily for the four models for which there were adequate data, and used only with caution for other models. These PEC can be adjusted within acceptable bounds based on additional considerations, such as quality and quantity of available measured data, spatial and temporal scales, and project scope and magnitude, and updated based on the framework presented herein. This initial meta-analysis sets the stage for more comprehensive meta-analysis to revise PEC as new PMs and more data become available. Keywords. Guidelines, Model calibration and validation, Performance measures and evaluation criteria. ydrologic and water quality (H/WQ) models are increasingly being used to determine the impacts of land management, land use, climate, and conservation practices on water resources, ecology, and water-related ecosystem services. Hydrologic cycle components and fate and transport of sediments and chemicals are examples of complex systems comprised of many processes that can be simulated using H/WQ models.

Improved APEX Model Simulation of Buffer Water Quality Benefits at Field Scale

“…3 and 4), with model performance statistics well within the acceptable limits for r 2 , NSE, and Pbias values set for this study and the higher PEC proposed by Moriasi et al (2015) for watershed simulation at a monthly time step. Sediment model performance indicators were acceptable only for the NB model ( fig.…”

“…The PEC used in this study for event-based simulations were r 2 ≥ 0.5 and NSE ≥ 0.3 for all outputs, and |Pbias| ≤35% for runoff, ≤60% for sediment, and ≤70% for TP. These criteria were justified because of the event temporal scale of this study, which is a more refined temporal resolution than the monthly temporal scale often considered (Moriasi et al, 2015;Wang et al, 2012). These PEC are typically indicative of a model developed for planning purposes, which would be directionally accurate.…”

Cost-effectiveness of reverse auctions for watershed nutrient reductions in the presence of climate variability: An empirical approach for the Boone River watershed

“…The statistical comparisons between the simulated sediment, NO 3 -, and total P loads were again generally quite strong. Overall, the pollutant calibration statistical results meet the criteria suggested by Moriasi et al (2015).…”

“…The resulting R 2 and NSE values ranged from 0.75 to 0.97 across the daily, monthly, and annual streamflow comparisons for the three simulation periods, and all of the statistics exceeded 0.9 for the monthly and annual comparisons. These results strongly exceed the successful hydrological model testing criteria suggested by Moriasi et al (2007Moriasi et al ( , 2015.…”

Hydrologic and Water Quality Models: Key Calibration and Validation Topics

“…A detailed discussion on model performance measures and criteria is provided by Moriasi et al (2015). Performance measures and evaluation criteria are important aspects of H/WQ modeling and should be set before C/V (ASCE, 1993;USEPA, 2002).…”