The Model Evaluation Tools (MET): More than a Decade of Community-Supported Forecast Verification

Brown, Barbara G.; Jensen, Tara; Gotway, John Halley; Bullock, Randy; Gilleland, Eric; Fowler, Tressa L.; Newman, Kathryn M.; Adriaansen, Dan; Blank, Lindsay; Burek, Tatiana; Harrold, Michelle; Hertneky, Tracy; Kalb, Christina; Kucera, Paul A.; Nance, Louisa; Opatz, John; Vigh, Jonathan L.; Wolff, Jamie K.

doi:10.1175/bams-d-19-0093.1

Cited by 41 publications

(36 citation statements)

References 59 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…All comparisons were made using NCAR's Model Evaluation Tools V9.0 (MET) package (Brown et al, 2020), utilizing a nearest-grid cell approach on an hourly temporal resolution.…”

Section: Verification and Diagnosticsmentioning

confidence: 99%

“…The namelist.input file, which is used for the WRF configuration, and scripts for running WRF in NWP mode are uploaded with open access to Zenodo: https://doi.org/10.5281/zenodo.3894491 Model Evaluation Tools V9.0 (MET) from the NCAR Research Applications Laboratory (generation of verification statistics) are open source and available from https://ral.ucar. edu/solutions/products/model-evaluation-tools-met(Brown et al, 2020). The NCAR Command Language (NCL) V6.2 (2019) consists of open-source graphics and is used for overwriting soil moisture data when running NWP mode.…”

mentioning

confidence: 99%

See 1 more Smart Citation

Seasonal and diurnal performance of daily forecasts with WRF V3.8.1 over the United Arab Emirates

et al. 2021

View full text Add to dashboard Cite

Abstract. Effective numerical weather forecasting is vital in arid regions like the United Arab Emirates (UAE) where extreme events like heat waves, flash floods, and dust storms are severe. Hence, accurate forecasting of quantities like surface temperatures and humidity is very important. To date, there have been few seasonal-to-annual scale verification studies with WRF at high spatial and temporal resolution. This study employs a convection-permitting scale (2.7 km grid scale) simulation with WRF with Noah-MP, in daily forecast mode, from 1 January to 30 November 2015. WRF was verified using measurements of 2 m air temperature (T2 m), 2 m dew point (TD2 m), and 10 m wind speed (UV10 m) from 48 UAE WMO-compliant surface weather stations. Analysis was made of seasonal and diurnal performance within the desert, marine, and mountain regions of the UAE. Results show that WRF represents temperature (T2 m) quite adequately during the daytime with biases ≤+1 ∘C. There is, however, a nocturnal cold bias (−1 to −4 ∘C), which increases during hotter months in the desert and mountain regions. The marine region has the smallest T2 m biases (≤-0.75 ∘C). WRF performs well regarding TD2 m, with mean biases mostly ≤ 1 ∘C. TD2 m over the marine region is overestimated, though (0.75–1 ∘C), and nocturnal mountain TD2 m is underestimated (∼-2 ∘C). UV10 m performance on land still needs improvement, and biases can occasionally be large (1–2 m s−1). This performance tends to worsen during the hot months, particularly inland with peak biases reaching ∼ 3 m s−1. UV10 m is better simulated in the marine region (bias ≤ 1 m s−1). There is an apparent relationship between T2 m bias and UV10 m bias, which may indicate issues in simulation of the daytime sea breeze. TD2 m biases tend to be more independent. Studies such as these are vital for accurate assessment of WRF nowcasting performance and to identify model deficiencies. By combining sensitivity tests, process, and observational studies with seasonal verification, we can further improve forecasting systems for the UAE.

show abstract

“…All comparisons were made using NCAR's Model Evaluation Tools V9.0 (MET) package (Brown et al, 2020), utilizing a nearest-grid cell approach on an hourly temporal resolution.…”

Section: Verification and Diagnosticsmentioning

confidence: 99%

mentioning

confidence: 99%

Seasonal and diurnal performance of daily forecasts with WRF V3.8.1 over the United Arab Emirates

et al. 2021

View full text Add to dashboard Cite

show abstract

“…In fact, any forecast that correctly predicts the occurrence of highly localized heavy rain, may incur the so called "double penalty" error [90] if it places the event in a nearby area, producing, for example, a root mean squared error (RMSE, see [91]) higher than another forecast which completely misses the prediction. To overcome such a limitation, object-based verification methods have been developed by the scientific community [92], and currently, software packages for practical applications exist [93]. These methods exhibit some drawbacks, such as the smoothing and filtering the observations undergo, and the large number of parameters whose settings are somewhat arbitrary.…”

Section: Quantitative Precipitation Forecast Verificationmentioning

confidence: 99%

Assimilating X- and S-Band Radar Data for a Heavy Precipitation Event in Italy

Capecchi¹,

Antonini²,

Benedetti³

et al. 2021

Water

View full text Add to dashboard Cite

During the night between 9 and 10 September 2017, multiple flash floods associated with a heavy-precipitation event affected the town of Livorno, located in Tuscany, Italy. Accumulated precipitation exceeding 200 mm in two hours was recorded. This rainfall intensity is associated with a return period of higher than 200 years. As a consequence, all the largest streams of the Livorno municipality flooded several areas of the town. We used the limited-area weather research and forecasting (WRF) model, in a convection-permitting setup, to reconstruct the extreme event leading to the flash floods. We evaluated possible forecasting improvements emerging from the assimilation of local ground stations and X- and S-band radar data into the WRF, using the configuration operational at the meteorological center of Tuscany region (LaMMA) at the time of the event. Simulations were verified against weather station observations, through an innovative method aimed at disentangling the positioning and intensity errors of precipitation forecasts. A more accurate description of the low-level flows and a better assessment of the atmospheric water vapor field showed how the assimilation of radar data can improve quantitative precipitation forecasts.

show abstract

“…In fact, any forecast that correctly predicts the occurrence of a highly localized heavy rain, may incur in the so called "double penalty" error [81] if it places the event in a nearby area, producing, for example, a root mean squared error (RMSE, see [82]) higher than another forecast which completely misses the prediction. To overcome such a limitation, object-based verification methods have been developed by the scientific community [83] and, currently, software packages for practical applications exist [84]. Anyway these methods exhibit some drawbacks, like the smoothing and filtering the observations undergo and the large number of parameters whose setting turns out somewhat arbitrary.…”

Section: Quantitative Precipitation Forecast Verificationmentioning

confidence: 99%

Assimilating X- and S-band Radar Data for a Heavy Precipitation Event in Italy

Capecchi¹,

Antonini²,

Benedetti³

et al. 2021

Preprint

View full text Add to dashboard Cite

During the night between 9 and 10 September 2017, multiple flash floods associated to a heavy-precipitation event affected the town of Livorno, located in Tuscany, Italy. Accumulated precipitation exceeding 200 mm in two hours, associated with a return period higher than 200 years, caused all the largest streams of the Livorno municipality to flood several areas of the town. We used the limited-area Weather Research and Forecasting (WRF) model, in a convection-permitting setup, to reconstruct the extreme event leading to the flash floods. We evaluated possible forecasting improvements emerging from the assimilation of local ground stations and X- and S-band radar data into the WRF, using the configuration operational at the meteorological center of Tuscany region (LaMMA) at the time of the event. Simulations were verified against weather station observations, through an innovative method aimed at disentangling the positioning and intensity errors of precipitation forecasts. By providing more accurate descriptions of the low-level flow and a better assessment of the atmospheric water vapour, the results demonstrate that assimilating radar data improved the quantitative precipitation forecasts.

show abstract

The Model Evaluation Tools (MET): More than a Decade of Community-Supported Forecast Verification

Abstract: Capsule summary MET is a community-based package of state-of-the-art tools to evaluate predictions of weather, climate, and other phenomena, with capabilities to display and analyze verification results via the METplus system.

Cited by 41 publications

References 59 publications

Seasonal and diurnal performance of daily forecasts with WRF V3.8.1 over the United Arab Emirates

Seasonal and diurnal performance of daily forecasts with WRF V3.8.1 over the United Arab Emirates

Assimilating X- and S-Band Radar Data for a Heavy Precipitation Event in Italy

Assimilating X- and S-band Radar Data for a Heavy Precipitation Event in Italy

Contact Info

Product

Resources

About