Network inference using directed information: The deterministic limit

Rahimzamani, Arman; Kannan, Sreeram

doi:10.1109/allerton.2016.7852224

“…This was originally studied in the setting when the variables are jointly Gaussian and hence the dependence is linear (see [6] for the original treatment, and [7,8] for versions with latent variables). This problem was generalized to the setting with arbitrary probability distributions and temporal dependences in [9] and studied further in [10], for one-step markov chains in [11] and deterministic relationships in [12]. From these works, under some technical condition, we can assert that the following method is guaranteed to be consistent,…”

Section: )mentioning

confidence: 99%

Potential conditional mutual information: Estimators and properties

Rahimzamani

¹

,

Kannan

²

2017

Preprint

Self Cite

View full text Add to dashboard Cite

The conditional mutual information I(X; Y |Z) measures the average information that X and Y contain about each other given Z. This is an important primitive in many learning problems including conditional independence testing, graphical model inference, causal strength estimation and time-series problems. In several applications, it is desirable to have a functional purely of the conditional distribution p Y |X,Z rather than of the joint distribution pX,Y,Z . We define the potential conditional mutual information as the conditional mutual information calculated with a modified joint distribution p Y |X,Z qX,Z , where qX,Z is a potential distribution, fixed airport. We develop K nearest neighbor based estimators for this functional, employing importance sampling, and a coupling trick, and prove the finite k consistency of such an estimator. We demonstrate that the estimator has excellent practical performance and show an application in dynamical system inference.

show abstract

“…Non-parametric inference of the causal influence structure underlying a set of observed time-series was studied in [6] where it was shown that, under mild assumptions, this structure can be accurately reconstructed from the observed time-series if the number of available samples in each time-series is large enough. Similarly to the ideas presented in [7], [6] also proposed to evaluate the causal influence between two time-series when statistically conditioning on the rest of the observed timeseries. As a measure for causal influence, [6] used the directed information (DI) functional (which is closely related to the transfer entropy (TE) functional).…”

Section: Introductionmentioning

confidence: 99%

“…Similarly to the ideas presented in [7], [6] also proposed to evaluate the causal influence between two time-series when statistically conditioning on the rest of the observed timeseries. As a measure for causal influence, [6] used the directed information (DI) functional (which is closely related to the transfer entropy (TE) functional). Noting that the DI must be estimated from the observed time-series, conditioning on the rest of the time-series significantly enlarged the effective statespace in the estimation problem; even when the number of time-series is moderate, this task requires a huge number of samples.…”

Section: Introductionmentioning

confidence: 99%

“…Noting that the DI must be estimated from the observed time-series, conditioning on the rest of the time-series significantly enlarged the effective statespace in the estimation problem; even when the number of time-series is moderate, this task requires a huge number of samples. As the number of available samples in the problem studied in the current work is relatively small, we do not try to infer the underlying causal structure as in [6]. Instead of estimating the DI by conditioning on the rest of the time-series, we estimate the non-conditioned pair-wise DI, which requires significantly fewer samples.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Ranking causal influence of financial markets via directed information graphs

Diamandis

¹

,

Murin

²

,

Goldsmith

³

2018

2018 52nd Annual Conference on Information Sciences and Systems (CISS)

View full text Add to dashboard Cite

A non-parametric method for ranking stock indices according to their mutual causal influences is presented. Under the assumption that indices reflect the underlying economy of a country, such a ranking indicates which countries exert the most economic influence in an examined subset of the global economy. The proposed method represents the indices as nodes in a directed graph, where the edges' weights are estimates of the pair-wise causal influences, quantified using the directed information functional. This method facilitates using a relatively small number of samples from each index. The indices are then ranked according to their net-flow in the estimated graph (sum of the incoming weights subtracted from the sum of outgoing weights). Daily and minute-by-minute data from nine indices (three from Asia, three from Europe and three from the US) were analyzed. The analysis of daily data indicates that the US indices are the most influential, which is consistent with intuition that the indices representing larger economies usually exert more influence. Yet, it is also shown that an index representing a small economy can strongly influence an index representing a large economy if the smaller economy is indicative of a larger phenomenon. Finally, it is shown that while interregion interactions can be captured using daily data, intra-region interactions require more frequent samples.

show abstract

“…This was originally studied in the setting when the variables are jointly Gaussian and hence the dependence is linear (see [6] for the original treatment, and [7,8] for versions with latent variables). This problem was generalized to the setting with arbitrary probability distributions and temporal dependences in [9] and studied further in [10], for one-step markov chains in [11] and deterministic relationships in [12]. From these works, under some technical condition, we can assert that the following method is guaranteed to be consistent,…”

Section: Introductionmentioning

confidence: 99%

Potential conditional mutual information: Estimators and properties

Rahimzamani

¹

,

Kannan

²

2017

Preprint

Self Cite

View full text Add to dashboard Cite

The conditional mutual information I(X; Y |Z) measures the average information that X and Y contain about each other given Z. This is an important primitive in many learning problems including conditional independence testing, graphical model inference, causal strength estimation and time-series problems. In several applications, it is desirable to have a functional purely of the conditional distribution p Y |X,Z rather than of the joint distribution pX,Y,Z . We define the potential conditional mutual information as the conditional mutual information calculated with a modified joint distribution p Y |X,Z qX,Z , where qX,Z is a potential distribution, fixed airport. We develop K nearest neighbor based estimators for this functional, employing importance sampling, and a coupling trick, and prove the finite k consistency of such an estimator. We demonstrate that the estimator has excellent practical performance and show an application in dynamical system inference.

show abstract

Network inference using directed information: The deterministic limit

Cited by 12 publications

References 16 publications

Potential conditional mutual information: Estimators and properties

Potential conditional mutual information: Estimators and properties

Ranking causal influence of financial markets via directed information graphs

Potential conditional mutual information: Estimators and properties

Contact Info

Product

Resources

About