The joint influence of break and noise variance on the break detection capability in time series homogenization

Abstract

Abstract. Instrumental climate records of the last centuries suffer from multiple breaks due to relocations and changes in measurement techniques. These breaks are detected by relative homogenization algorithms using the difference time series between a candidate and a reference. Modern multiple changepoint methods use a decomposition approach where the segmentation explaining most variance defines the breakpoints, while a stop criterion restricts the number of breaks. In this study a pairwise multiple breakpoint algorithm consisting of these two components is tested with simulated data for a range of signal-to-noise ratios (SNRs) found in monthly temperature station datasets. The results for low SNRs obtained by this algorithm do not differ much from random segmentations; simply increasing the stop criterion to reduce the number of breaks is shown to not be helpful. This can be understood by considering that, in case of multiple breakpoints, even a random segmentation can explain about half of the break variance. We derive analytical equations for the explained noise and break variance for random and optimal segmentations. From these we conclude that reliable break detection at low but realistic SNRs needs a new approach. The problem is relevant because the uncertainty of the trends of individual stations is shown to be climatologically significant also for these small SNRs. An important side result is a new method to determine the break variance and the number of breaks in a difference time series by studying the explained variance for random break positions. We further discuss the changes from monthly to annual scale which increase the SNR by more than a factor of 3.

Homogenizing GPS Integrated Water Vapor Time Series: Benchmarking Break Detection Methods on Synthetic Data Sets

“…Using very high time frequency data series increases, for example, the noise of time series. Lindau and Venema (2018) showed that for a pairwise multiple breakpoint algorithm, the results for low signal‐to‐noise ratios (SNRs) do not differ much from random segmentations and that reliable break detection at low but realistic SNRs needs a new approach. However, a break identified by on the methods assessed here can be adjusted for in the individual observation series, and these homogenized individual data points can then be used for weather and climate extreme applications and assimilation into reanalysis products.…”

A new method to study inhomogeneities in climate records: Brownian motion or random deviations?

“…Thus, to decide which number of breaks is optimal a stop criterion is needed that penalizes the insertion of breaks. Caussinus and Mestre (2004), Domonkos (2011a) and Lindau and Venema (2018a) use the Lyazrhi stop criterion (Caussinus and Lyazrhi, 1997), which is given by…”

“…For the ideal case without noise, Lindau and Venema (2018a) showed that the explained variance grows with k by…”

A new method to study inhomogeneities in climate records: Brownian Motion or Random Deviations?

“…This suggests that stairs are easier to detect than platforms, although also gradual inhomogeneities modelled as linear trends in the station data could have lowered the percentage of detected platforms. The percentage of stairs and platforms as they are detectable by a homogenization algorithm will be further dependent, e.g., on the signal-to-noise ratio (SNR), which is shown to be a key parameter for break detection (Lindau and Venema, 2016;Lindau and Venema, 2018a). Thus, to conclude that the reduced number of detected platforms is directly caused by an admixture of BM-type breaks is risky and has been a motivation for this further study.…”