Assessing Granger Causality on Irregular Missing and Extreme Data

Abstract

The Granger test is one of the best known techniques to detect causality relationships among time series, and has been used uncountable times in science and engineering. The quality of its results strongly depends on the quality of the underlying data, and different approaches have been proposed to reduce the impact of, for instance, observational noise or irregular sampling. Less attention has nevertheless been devoted to situations in which the analysed time series are irregularly polluted with missing and extreme values. In this contribution I tackle this problem by comparing four different data pre-processing strategies and evaluating their performance with synthetic time series, both in dyadic tests and functional network contexts. I further apply these strategies to a real-world problem, involving inferring the structure behind the propagation of delays in an air transport system. Finally, some guidelines are provided on when and how these strategies ought to be used.

Air delay propagation patterns in Europe from 2015 to 2018: an information processing perspective

“…Such discrepancies can be due to many factors, including the use of different data sets with different number of airports; the different geographical area considered in [40], which also implies different prioritisation rules for flights and hence delay mitigation strategies; and the way data are pre-processed, as discussed in [43]. This raises questions about the reproducibility and validation of results; beyond the analyses presented in section 5, this topic will be discussed in the next section.…”

“…This effectively implies that missing elements are excluded from the calculation of the autoregressive models, and that the Granger causality test is performed only on the values esteemed as correct. As discussed in [43], this improves the sensitivity of the Granger test even when a significant fraction of values are missing, and allows recovering most of the original causal relationships.…”

“…whether b is actually having an impact in the prediction. One important limitation of the Granger test is its sensitivity to missing values [43,51]. To illustrate, and considering the application at hand, many airports do not operate around the clock, such that some hours can have no operations associated to them, resulting in a zero average delay.…”

“…While this is a problem well-known in neuroscience literature [73,74], little is known about how delay propagation patterns are affected by the choice of the connectivity metric [37,41]. As shown in [43], even a same causality metric can yield different results according to how missing values are dealt with. It is further possible that a correct evaluation of these propagation structures will require tailored, yet to be developed metrics, on the line of what proposed in [41].…”

“…In this contribution we propose a large-scale analysis of the structure and evolution of the delay propagation network in Europe, by leveraging on the techniques developed in network neuroscience in the last decades, and on more recent instruments for data pre-processing and network analysis. More in details, we describe the delay propagation patterns across the 50 largest European airports during four years, from 2015 to 2018, using a modified version of the celebrated Granger causality to compensate for the presence of missing values [43]. The resulting functional networks are analysed using standard network science metrics, to describe how the topology has changed through time in response to external events.…”