Improved estimators of correlation and R² for skewed hydrologic data

Abstract

The coefficient of determination R 2 and Pearson correlation coefficient ρ = R are standard metrics in hydrology for the evaluation of the goodness of fit between model simulations and observations, and as measures of the degree of dependence of one variable upon another. We show that the standard product moment estimator of ρ, termed r, while well-behaved for bivariate normal data, is upward biased and highly variable for bivariate non-normal data. We introduce three alternative estimators of ρ which are nearly unbiased and exhibit much less variability than r for non-normal data. We also document remarkable upward bias and tremendous increases in variability associated with r using both synthetic data and daily streamflow simulations from 905 calibrated rainfall-runoff models. We show that estimators of ρ = R accounting for skewness are needed for daily streamflow series because they exhibit high variability and skewness compared to, for example, monthly/annual series, where r should perform well.

Improved Estimators of Model Performance Efficiency for Skewed Hydrologic Data

“…A Weibull plotting position is suitable here, because it yields an unbiased estimate of the cumulative probability associated with Figure 3 uses boxplots and a scatterplot to summarize the square of the PPCC values associated with the BLN3-MM model (denoted PPCC 2 m ) versus the PPCC 2 value of fitting a single LN3 model (denoted PPCC LN3 ) to the entire n day series at the 447 sites. Figure 3 documents the considerable improvement in the goodness of fit of the 48-parameter BLN3-MM model over the four-parameter BLN3 model used by Barber et al (2019), which is expected given the 44 additional parameters associated with the BLN3-MM model.…”

“…Daily, hourly, and subhourly streamflow are known to exhibit extremely high values of skewness, so that typical observations O, and simulations S, are much more closely approximated by a bivariate three-parameter lognormal (BLN3) model, than a bivariate normal model as was shown by Barber et al (2019) and others. Barber et al (2019), Blum et al (2017), and Limbrunner et al (2000, Figure 6) used L-moment diagrams to illustrate that two-and three-parameter lognormal distributions (LN2 and LN3, respectively) provide a good approximation to the pd of daily streamflow observations for hundreds of stations across the conterminous United States. Barber et al (2019) also document that the BLN3 model is equivalent to a Gaussian copula with a LN3 marginal pd.…”

“…Previous studies used actual streamflow observations and hydrologic model simulations to evaluate the performance of NSE and its variants, yet such studies can never report definitively on the performance of NSE because the true value of NSE, which we denote as E, is always unknown in such situations. Our approach of distinguishing between E and NSE enables implementation of controlled Monte Carlo experiments to rigorously evaluate the performance of various alternative estimators of efficiency E. Finally, we test our findings and recommendations using the output of hundreds of calibrated U.S. Geological Survey (USGS) Precipitation Runoff Modeling System (PRMS) rainfall-runoff models (Markstrom et al, 2015) analogous to the recent work of Farmer and Vogel (2016a) and Barber et al (2019).…”

“…Barber et al (2019), Blum et al (2017), and Limbrunner et al (2000, Figure 6) used L‐moment diagrams to illustrate that two‐ and three‐parameter lognormal distributions ( LN2 and LN3 , respectively) provide a good approximation to the pd of daily streamflow observations for hundreds of stations across the conterminous United States. Barber et al (2019) also document that the BLN3 model is equivalent to a Gaussian copula with a LN3 marginal pd.…”

“…Our approach of distinguishing between E and NSE enables implementation of controlled Monte Carlo experiments to rigorously evaluate the performance of various alternative estimators of efficiency E . Finally, we test our findings and recommendations using the output of hundreds of calibrated U.S. Geological Survey (USGS) Precipitation Runoff Modeling System (PRMS) rainfall‐runoff models (Markstrom et al, 2015) analogous to the recent work of Farmer and Vogel (2016a) and Barber et al (2019).…”