Economy Statistical Recurrent Units For Inferring Nonlinear Granger Causality

Abstract

Granger causality is a widely-used criterion for analyzing interactions in largescale networks. As most physical interactions are inherently nonlinear, we consider the problem of inferring the existence of pairwise Granger causality between nonlinearly interacting stochastic processes from their time series measurements. Our proposed approach relies on modeling the embedded nonlinearities in the measurements using a component-wise time series prediction model based on Statistical Recurrent Units (SRUs). We make a case that the network topology of Granger causal relations is directly inferrable from a structured sparse estimate of the internal parameters of the SRU networks trained to predict the processes' time series measurements. We propose a variant of SRU, called economy-SRU, which, by design has considerably fewer trainable parameters, and therefore less prone to overfitting. The economy-SRU computes a low-dimensional sketch of its high-dimensional hidden state in the form of random projections to generate the feedback for its recurrent processing. Additionally, the internal weight parameters of the economy-SRU are strategically regularized in a group-wise manner to facilitate the proposed network in extracting meaningful predictive features that are highly time-localized to mimic real-world causal events. Extensive experiments are carried out to demonstrate that the proposed economy-SRU based time series prediction model outperforms the MLP, LSTM and attention-gated CNN-based time series models considered previously for inferring Granger causality.

Neural Additive Vector Autoregression Models for Causal Discovery in Time Series

“…Similar to the models in (Khanna and Tan 2019), we assume a maximum lag of 2 for the MLP models, use 10 hidden units per layer, and ignore self-links in the AUROC calculation. The other hyperparameters are tuned using a 80/20% training/validation split, where we train on the first 80% of timesteps, and select the hyperparameters with lowest mean squared error on the final 20% time steps.…”

“…The selected hyperparameters are reported in Appendix A. The hyperparameters of the other neural models are tuned in tanta- mount manner and can be found in (Khanna and Tan 2019) Appendix G. We report the average AUROC over 100 different runs of the NAVAR model.…”

“…(Marinazzo et al 2011;Stephan et al 2008)) relatively few others (e.g. (Tank et al 2018;Khanna and Tan 2019)), have harnessed the power of deep learning for causal discovery in time series. These methods operate within the Granger causality framework and use deep neural networks to model the time dependencies and interactions between the variables.…”

“…Average AUROC on the DREAM3 gene expression dataset. Neural methods are indicated with an asterisk, and their scores are obtained from(Khanna and Tan 2019). Models with the highest AUROC are indicated in boldface .…”

Deep Recurrent Modelling of Granger Causality with Latent Confounding

“…Model-free approaches such as transfer entropy (Vicente et al, 2011) are able to detect nonlinear dependencies between time series, however they suffer from high variance and require large amounts of data for reliable estimation (Tank et al, 2021). In this work, we follow a recent trend that uses neural networks to infer complex nonlinear causal dependencies in time series data (Khanna & Tan, 2020;Nauta et al, 2019;Tank et al, 2021;Bussmann et al, 2020;Trifunov et al, 2019;De Brouwer et al, 2020;Marcinkevičs & Vogt, 2021;Moraffah et al, 2021).…”