Data‐based comparison of frequency analysis methods: A general framework

Abstract

[1] An abundance of methods have been developed over the years to perform the frequency analysis (FA) of extreme environmental variables. Although numerous comparisons between these methods have been implemented, no general comparison framework has been agreed upon so far. The objective of this paper is to build the foundation of a data-based comparison framework, which aims at complementing more standard comparison schemes based on Monte Carlo simulations or statistical testing. This framework is based on the following general principles: (i) emphasis is put on the predictive ability of competing FA implementations, rather than their sole descriptive ability measured by some goodness-of-fit criterion; (ii) predictive ability is quantified by means of reliability indices, describing the consistency between validation data (not used for calibration) and FA predictions; (iii) stability is also quantified, i.e., the ability of a FA implementation to yield similar estimates when calibration data change; and (iv) the necessity to subject uncertainty estimates to the same scrutiny as point estimates is recognized, and a practical approach based on the use of the predictive distribution is proposed for this purpose. This framework is then applied to a case study involving 364 gauging stations in France, where 10 FA implementations are compared. These implementations correspond to the local, regional, and local-regional estimation of Gumbel and generalized extreme value distributions. Results show that reliability and stability indices are able to reveal marked differences between FA implementations. Moreover, the case study also confirms that using the predictive distribution to indirectly scrutinize uncertainty estimates is a viable approach, with distinct FA implementations showing marked differences in the reliability of their uncertainty estimates. The proposed comparison framework therefore constitutes a valuable tool to compare the predictive reliability of competing FA implementations, along with the reliability of their uncertainty estimates.

Evaluation of a compound distribution based on weather pattern subsampling for extreme rainfall in Norway

“…We follow the split sample evaluation proposed in Garavaglia et al (2011) and Renard et al (2013). We divide the data for each station i into two subsamples, C…”

“…For this, three scores are computed, assessing respectively stability (SPAN) and reliability (AREA(FF) and AREA(N T )) of the fits. These scores were proposed and used in Garavaglia et al (2011) and Renard et al (2013). The SPAN criterion evaluates the stability of the return level estimation, when using data for each of the two subsamples.…”

“…Hypothesis testing for assessing if the uniform assumption is valid is challenging because the ff i are not independent from site to site, due to the spatial dependence between data. Thus Renard et al (2013) proposed to base comparison on the graphical analysis of cumulative distribution functions (CDFs), by inspecting how much the CDF of the ff diverge from the 1 : 1 line, corresponding to the CDF of uniform variates on (0, 1). A quantitative assessment of this divergence is provided by computing the area between both CDFs.…”

La méthode SHYREG débit—application sur 1605 bassins versants en France métropolitaine

“…La stabilité est définie comme la capacité d'une méthode à fournir des estimations de quantiles de débit à des fréquences rares et extrêmes peu sensibles au nombre d'années d'observations servant au calage de la méthode. La stabilité est étudiée à travers le critère SPAN T (Garavaglia et al 2011, Renard et al 2013. Les chroniques des débits observés qui disposent de 40 ans d'observation (soit 432 stations hydrométriques) sont découpées en deux périodes de 20 ans, notée C 1 et C 2 .…”

A flash flood hazard assessment in dry valleys (northern France) by cellular automata modelling

“…Two choices were possible: either we implement rains according to the frequency analysis methods for extremes, i.e. the SHYREG database (Renard et al 2013), or we consider project rainfall scenarios for all basins. The first choice was not appropriate for this study for two reasons: (1) the statistic calculation of rains presenting small probability and large return periods introduce high uncertainties; (2) earlier research (Douvinet et al 2009) underlined differences between measurements by official stations, radar, and volunteer stations.…”