Application of Forecast Verification Science to Operational River Forecasting in the U.S. National Weather Service

Demargne, Julie; Mullusky, Mary; Werner, Kevin; Adams, Thomas E.; Lindsey, S.; Schwein, Noreen; Marosi, William; Welles, E.

doi:10.1175/2008bams2619.1

Cited by 42 publications

(29 citation statements)

References 2 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Verification of such probabilistic forecasts is a particular challenge (Demargne et al, 2009;Cloke and Pappenberger, 2009;Demargne et al, 2010;Brown et al, 2010), in that 'right and wrong' no longer have well-defined meanings when it comes to a single forecast observation pair, or a single Some open questions remain on the need for specific components for hydrological applications and how far the "meteorological way of doing" is adequate / applicable for hydrological questions. Schaake et al (2007a) describe a technique, used at the US National Weather Service, for generating an ensemble from a single-valued forecast of precipitation and temperature.…”

Section: Accepted M Manuscriptmentioning

confidence: 99%

The COST 731 Action: A review on uncertainty propagation in advanced hydro-meteorological forecast systems

Rossa¹,

Liechti

Zappa

et al. 2011

Atmospheric Research

View full text Add to dashboard Cite

Publication informationAtmospheric Research, 100 (2-3): 150-167Publisher Elsevier This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain. A C C E P T E D M A N U S C R I P T ACCEPTED MANUSCRIPT A C C E P T E D M A N U S C R I P T ACCEPTED MANUSCRIPT AbstractQuantifying uncertainty in flood forecasting is a difficult task, given the multiple and strongly nonlinear model components involved in such a system. Much effort has been and is being invested in the quest of dealing with uncertain precipitation observations and forecasts and the propagation of such uncertainties through hydrological and hydraulic models predicting river discharges and risk for inundation. The COST 731 Action is one of these and constitutes a European initiative which deals with the quantification of forecast uncertainty in hydro-meteorological forecast systems.COST 731 addresses three major lines of development: (1) combining meteorological and hydrological models to form a forecast chain, (2) propagating uncertainty information through this chain and make it available to end users in a suitable form, (3)

show abstract

Section: Accepted M Manuscriptmentioning

confidence: 99%

The COST 731 Action: A review on uncertainty propagation in advanced hydro-meteorological forecast systems

Rossa¹,

Liechti

Zappa

et al. 2011

Atmospheric Research

View full text Add to dashboard Cite

show abstract

“…The forecasts from these operational systems are evaluated in terms of the degree of similarity between some verification data, such as observations of river discharge, and the forecast (Demargne et al, 2009). However, another important component of the forecast evaluation is whether the forecasts add value or have skill compared to climatology or another simple 'best guess' (Luo et al, 2012;Perrin et al, 2006;Fewtrell et al, 2011).…”

Section: Introductionmentioning

confidence: 99%

How do I know if my forecasts are better? Using benchmarks in hydrological ensemble prediction

Pappenberger

Ramos²,

Cloke

et al. 2015

Journal of Hydrology

168

152

View full text Add to dashboard Cite

s u m m a r yThe skill of a forecast can be assessed by comparing the relative proximity of both the forecast and a benchmark to the observations. Example benchmarks include climatology or a naïve forecast. Hydrological ensemble prediction systems (HEPS) are currently transforming the hydrological forecasting environment but in this new field there is little information to guide researchers and operational forecasters on how benchmarks can be best used to evaluate their probabilistic forecasts. In this study, it is identified that the forecast skill calculated can vary depending on the benchmark selected and that the selection of a benchmark for determining forecasting system skill is sensitive to a number of hydrological and system factors. A benchmark intercomparison experiment is then undertaken using the continuous ranked probability score (CRPS), a reference forecasting system and a suite of 23 different methods to derive benchmarks. The benchmarks are assessed within the operational set-up of the European Flood Awareness System (EFAS) to determine those that are 'toughest to beat' and so give the most robust discrimination of forecast skill, particularly for the spatial average fields that EFAS relies upon.Evaluating against an observed discharge proxy the benchmark that has most utility for EFAS and avoids the most naïve skill across different hydrological situations is found to be meteorological persistency. This benchmark uses the latest meteorological observations of precipitation and temperature to drive the hydrological model. Hydrological long term average benchmarks, which are currently used in EFAS, are very easily beaten by the forecasting system and the use of these produces much naïve skill. When decomposed into seasons, the advanced meteorological benchmarks, which make use of meteorological observations from the past 20 years at the same calendar date, have the most skill discrimination. They are also good at discriminating skill in low flows and for all catchment sizes. Simpler meteorological benchmarks are particularly useful for high flows. Recommendations for EFAS are to move to routine use of meteorological persistency, an advanced meteorological benchmark and a simple meteorological benchmark in order to provide a robust evaluation of forecast skill. This work provides the first comprehensive evidence on how benchmarks can be used in evaluation of skill in probabilistic hydrological forecasts and which benchmarks are most useful for skill discrimination and avoidance of naïve skill in a large scale HEPS. It is recommended that all HEPS use the evidence and methodology provided here to evaluate which benchmarks to employ; so forecasters can have trust in their skill evaluation and will have confidence that their forecasts are indeed better.

show abstract

“…The reliability and accuracy of hydrologic predictions are negatively affected by a number of sources, including unreliable and inaccurate meteorological forcing, initial basin state conditions, model errors (i.e., incorrect model structure and parameter specification), and errors in streamflow observations (McEnery et al, 2005;Schaake et al, 2006). In the ESP system like the US National Weather Service River Forecast System (Seo et al, 2006;DeMargne et al, 2009), these uncertainties are handled by different statistical processors (Fig. 1).…”

Section: Introductionmentioning

confidence: 99%

A hydrologic post-processor for ensemble streamflow predictions

et al. 2011

View full text Add to dashboard Cite

Abstract. This paper evaluates the performance of a statistical post-processor for imperfect hydrologic model forecasts. Assuming that the meteorological forecasts are wellcalibrated, we employ a "General Linear Model (GLM)" to post-process simulations produced by a hydrologic model. For a particular forecast date, the observations and simulations from an "analysis window" and hydrologic model forecasts for a "forecast window", the GLM Post-Processor (GLMPP) is used to produce an ensemble of predictions of the streamflow observations that will occur during the "forecast window". The objectives of the GLMPP are to: (1) preserve any skill in the original hydrologic ensemble forecast; (2) correct systematic model biases; (3) retain the equallikelihood assumption for the ensemble; (4) preserve temporal scale dependency relationships in streamflow hydrographs and the uncertainty in the predictions; and, (5) produce reliable ensemble predictions.Observed and simulated daily streamflow data from the Second Workshop on Model Parameter Estimation Experiment (MOPEX) are used to test how well these objectives are met when the GLMPP is applied to ensemble hydrologic forecasts driven by well calibrated meteorological forecasts. A 39-year hydrologic dataset from the French Broad basin is split into calibration and verification periods. The results show that the GLMPP built using data from the calibration period removes the mean bias when applied to hydrologic model simulations from both the calibration and verification periods. Probability distributions of the post-processed model simulations are shown to be closer to the climatological probability distributions of observed streamflow than the distributions of the unadjusted simulated flows. A number of Correspondence to: Q. Duan (qyduan@bnu.edu.cn) experiments with different GLMPP configurations were also conducted to examine the effects of different configurations for forecast and analysis window lengths on the robustness of the results.

show abstract

Application of Forecast Verification Science to Operational River Forecasting in the U.S. National Weather Service

Cited by 42 publications

References 2 publications

The COST 731 Action: A review on uncertainty propagation in advanced hydro-meteorological forecast systems

The COST 731 Action: A review on uncertainty propagation in advanced hydro-meteorological forecast systems

How do I know if my forecasts are better? Using benchmarks in hydrological ensemble prediction

A hydrologic post-processor for ensemble streamflow predictions

Contact Info

Product

Resources

About