Short communication: On the use of the Benjamini–Hochberg procedure in wavelet analysis and its inferior performance to the cumulative arc‐wise and area‐wise tests

Abstract

The Benjamini-Hochberg (BH) procedure is a well-recognized and powerful tool for controlling the number of statistical artefacts arising from the simultaneous testing of multiple hypotheses. However, its performance as a statistical tool in wavelet analysis is unknown. A series of experiments suggests that the BH method is too conservative and too frequently rejects truly significant features such as periodicities as insignificant. In contrast, the recently developed cumulative area-wise and arc-wise tests strongly control the false positive rate yet liberally deem truly significant features as significant. Besides cone of influence impacts, statistical inferences made from the cumulative area-wise and arc-wise tests change very little when time series are lengthened, promoting the reproducibility of results in the face of continually lengthening climate time series. There is no such guarantee for BH method results, as they depend on the chosen family of tests, which will continually change as time series lengthen. The findings from the theoretical experiments and practical applications to Susquehanna streamflow suggest that the cumulative arc-wise and area-wise tests should be the preferred methods for statistical hypothesis testing in wavelet analysis.