WARMF: Model Use, Calibration, and Validation

Herr, Joel; Chen, C. W.

doi:10.13031/2013.42249

Cited by 17 publications

(19 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Of the models within the Moriasi et al (2012) special collection, only a few provided PEC (table 3), including BASINS/HSPF (Duda et al, 2012), DRAINMOD (Skaggs et al, 2012), EPIC/APEX (Wang et al, 2012), KINE-ROS/AGWA (Goodrich et al, 2012), RZWQM2 (Ma et al, 2012), and WARMF (Herr and Chen, 2012). PEC from Moriasi et al (2007) were cited for SWAT (Arnold et al, 2012), SWIM3 (Huth et al, 2012), and WEPP (Flanagan et al, 2012).…”

Section: Synthesis Of Performance Measures and Evaluation Criteriamentioning

confidence: 99%

“…Harmel et al (2014), Bennett et al (2013), Krause et al (2005), and Coffey (2004) also provide a comprehensive list of statistical PMs. Although there are different ways to categorize PMs (Moriasi et al, 2007;Bennett et al, 2013), the PMs in this article (Herr and Chen, 2012) Good Model Performance Hydrology/flow <20% absolute error Nutrients <30% absolute error Phytoplankton and suspended sediment <50% absolute error [a] Values estimated from figure 4 (Duda et al, 2012).…”

Section: Statistical Performance Measuresmentioning

confidence: 99%

“…Modelers have used different PMs, including statistical, graphical, or a combination of both. For example, Herr and Chen (2012) preferred the use of absolute and relative error, while Huth et al (2012) recommended and used a variety of measures, including Nash-Sutcliffe efficiency (NSE; Nash and Sutcliffe, 1970) and the ratio of root mean square error (RMSE) and standard deviation of measured data (RSR; Moriasi et al, 2007). Commonly used graphical PMs include time series plots (e.g., van der Keur et al, 2001;Mutiti and Levy, 2010;Palosuo et al, 2011;Arnold et al, 2012;Herr and Chen, 2012;Huth et al, 2012), scatter plots (e.g., Palosuo et al, 2011;Herr and Chen, 2012), cumulative charts (e.g., Herr and Chen, 2012), and contour maps (e.g., Zheng et al, 2012).…”

mentioning

confidence: 99%

See 2 more Smart Citations

Hydrologic and Water Quality Models: Performance Measures and Evaluation Criteria

2015

Trans. ASABE

1,604

471

View full text Add to dashboard Cite

Performance measures (PMs) and corresponding performance evaluation criteria (PEC) are important aspects of calibrating and validating hydrologic and water quality models and should be updated with advances in modeling science. We synthesized PMs and PEC from a previous special collection, performed a meta-analysis of performance data reported in recent peer-reviewed literature for three widely published watershed-scale models (SWAT, HSPF, WARMF), and one field-scale model (ADAPT), and provided guidelines for model performance evaluation. Based on the synthesis, meta-analysis, and personal modeling experiences, we recommend coefficient of determination (R 2 ; in conjunction with gradient and intercept of the corresponding regression line), Nash Sutcliffe efficiency (NSE), index of agreement (d), root mean square error (RMSE; alongside the ratio of RMSE and standard deviation of measured data, RSR), percent bias (PBIAS), and several graphical PMs to evaluate model performance. We recommend that model performance can be judged "satisfactory" for flow simulations if monthly R 2 > 0.70 and d > 0.75 for field-scale models, and daily, monthly, or annual R 2 > 0.60, NSE > 0.50, and PBIAS ≤ ±15% for watershed-scale models. Model performance at the watershed scale can be evaluated as "satisfactory" if monthly R 2 > 0.40 and NSE > 0.45 and daily, monthly, or annual PBIAS ≤ ±20% for sediment; monthly R 2 > 0.40 and NSE > 0.35 and daily, monthly, or annual PBIAS ≤ ±30% for phosphorus (P); and monthly R 2 > 0.30 and NSE > 0.35 and daily, monthly, or annual PBIAS ≤ ±30% for nitrogen (N). For RSR, we recommend that previously published PEC be used as detailed in this article. We also recommend that these PEC be used primarily for the four models for which there were adequate data, and used only with caution for other models. These PEC can be adjusted within acceptable bounds based on additional considerations, such as quality and quantity of available measured data, spatial and temporal scales, and project scope and magnitude, and updated based on the framework presented herein. This initial meta-analysis sets the stage for more comprehensive meta-analysis to revise PEC as new PMs and more data become available. Keywords. Guidelines, Model calibration and validation, Performance measures and evaluation criteria. ydrologic and water quality (H/WQ) models are increasingly being used to determine the impacts of land management, land use, climate, and conservation practices on water resources, ecology, and water-related ecosystem services. Hydrologic cycle components and fate and transport of sediments and chemicals are examples of complex systems comprised of many processes that can be simulated using H/WQ models.

show abstract

Section: Synthesis Of Performance Measures and Evaluation Criteriamentioning

confidence: 99%

Section: Statistical Performance Measuresmentioning

confidence: 99%

mentioning

confidence: 99%

See 1 more Smart Citation

Hydrologic and Water Quality Models: Performance Measures and Evaluation Criteria

2015

Trans. ASABE

1,604

471

View full text Add to dashboard Cite

show abstract

“…The complex calibration strategy, which can better account for spatial biophysiochemical variations, may also help reduce the problem of equifinality because fewer model parameter sets will satisfy the calibration criteria at all calibration sites. Multi-site calibration, moving from upstream to downstream locations, was demonstrated for WARMF (Herr and Chen, 2012) and SWAT (White and Chaubey, 2005;Arnold et al, 2012). Variations of multi-site calibration have been used in most watershed scale models, e.g., BASINS/HSPF (Duda et al, 2012), MIKE SHE (Refsgaard, 1997;Jaber and Shukla, 2012), EPIC and APEX (Wang et al, 2012), and KINEROS/AGWA (Goodrich et al, 2012).…”

Section: Complex Strategymentioning

confidence: 99%

A Recommended Calibration and Validation Strategy for Hydrologic and Water Quality Models

2015

Trans. ASABE

167

View full text Add to dashboard Cite

Hydrologic and water quality (H/WQ) models are widely used to support site-specific environmental assessment, design, planning, and decision making. Calibration and validation (C/V) are fundamental processes used to demonstrate that an H/WQ model can produce suitable results in a particular application. However, the lack of comprehensive guidelines has led to the use of ad hoc, inconsistent, and incomplete C/V processes, which have made it difficult to interpret the myriad of published modeling studies, reduced the utility of many modeling applications, and slowed the advancement of H/WQ modeling. The objective of this article is to provide a generalized structure and process to assist modelers in developing a C/V strategy for H/WQ modeling applications. These best practice recommendations were developed based on an expansive review of the modeling literature, including a special collection of articles on H/WQ model calibration, validation, and use, as well as extensive discussion and debate among the authors. The model C/V recommendations include careful consideration, execution, and documentation of the following elements: (1) goals of model use, (2) data and parameters used in C/V, and (3) model C/V processes. Considerations in element 3 include the warm-up period, C/V strategy complexity, C/V process staging, spatiotemporal allocation of C/V comparison data, manual vs. automatic C/V, and additional diagnostics. Notable examples from the literature are provided for each strategy element. The comprehensive C/V strategy described herein will allow for better interpretation of future modeling studies, improved utility of modeling applications, and more systematic advancement of H/WQ models.

show abstract

“…Rapidly evolving technologies such as remote sensing (RS), which provides spectral data that can be interpreted to provide data on factors such as land use, evaporation and evapotranspiration, and soil moisture, are being directly supported by some of the more popular TMDL modeling tools such as BASINS (USEPA 2004) and watershed analysis risk management framework (WARMF) (Goldstein 2001;Herr and Chen 2012). However, outside these tools and their customized user interface for data assimilation, there are no standardized protocols for acquisition and processing of imagery.…”

Section: Geographic Information Systems and Remote Sensingmentioning

confidence: 99%

Tool for Searching USEPA’s TMDL Reports Repository to Analyze TMDL Modeling State of the Practice

et al. 2019

View full text Add to dashboard Cite

Total maximum daily load (TMDL) reports archived in the USEPA database can provide useful guidance for the development of new TMDLs and watershed management plans by aiding the selection process for the most appropriate modeling tool. The database contains more than 70,000 individual documents; therefore, a rapid screening tool is needed to elicit information about previous modeling studies that might help guide stakeholders and regulators in dealing with the TMDL application at hand, save time, and lead to a more cost-effective regulatory outcome. The paper introduces a smart web-based software tool for TMDL report selection based on different water management criteria. The tool uses an automated search method based on frequency of common water body impairments and models to categorize and select TMDL reports. Additionally, this tool provides better insight on the relationship between the modeling tools used and the impairments they address. This tool has proven useful in reviewing the state of integrated modeling (IM), applications of remote sensing (RS), application of basic versus mechanistic modeling, margin of safety (MOS) assessment, and the state of practice regarding relationships among impairments, models, and regions where TMDLs for various pollutants are being developed. Despite limitations on direct access to all TMDLs developed and reported to the EPA by the user, the tool can be improved over time to derive a better understanding of the relationships between these impairments, data, and the TMDL development process. Although the MOS is not directly quantified in the current version of the TRS tool, this feature may be incorporated in future updates.

show abstract

WARMF: Model Use, Calibration, and Validation

Cited by 17 publications

References 0 publications

Hydrologic and Water Quality Models: Performance Measures and Evaluation Criteria

Hydrologic and Water Quality Models: Performance Measures and Evaluation Criteria

A Recommended Calibration and Validation Strategy for Hydrologic and Water Quality Models

Tool for Searching USEPA’s TMDL Reports Repository to Analyze TMDL Modeling State of the Practice

Contact Info

Product

Resources

About