Permute-match tests: Detecting significant correlations between time series despite nonstationarity and limited replicates

Abstract

Many processes of scientific interest are nonstationary, meaning that they experience systematic changes over time. These processes pose a myriad of challenges to data analysis. One such challenge is the problem of testing for statistical dependence between two nonstationary time series. Existing tests mostly require strong modeling assumptions and/or are largely heuristic. If multiple independent and statistically identical replicates are available, a trial-swapping permutation test can be used. That is, within-replicate correlations (between time series of X and Y from the same replicate) can be compared to between-replicate correlations (between X from one replicate and Y from another). Although this method is simple and largely assumption-free, it is severely limited by the number of replicates. In particular, the lowest attainable p-value is 1/n! where n is the number of replicates. We describe a modified permutation test that partially alleviates this issue. Our test reports a lower p-value of 1/nnwhen there is particularly strong evidence of dependence, and otherwise defaults to a regular trial-swapping permutation test. We use this method to confirm the observation that groups of zebrafish swim faster when they are aligned, using an existing dataset with only 3 biological replicates.