Directed information, causal estimation, and communication in continuous time

Kim, Young-Han; Permuter, Haim H.; Weissman, Tsachy

doi:10.1109/wiopt.2009.5291562

Cited by 3 publications

References 43 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

To feed or not to feed back

Asnani¹,

Permuter²,

Weissman³

2011

2011 IEEE International Symposium on Information Theory Proceedings

View full text Add to dashboard Cite

We study the communication over Finite State Channels (FSCs), where the encoder and the decoder can control the availability or the quality of the noise-free feedback. Specifically, the instantaneous feedback is a function of an action taken by the encoder, an action taken by the decoder, and the channel output. Encoder and decoder actions take values in finite alphabets, and may be subject to average cost constraints.We prove capacity results for such a setting by constructing a sequence of achievable rates, using a simple scheme based on 'code tree' generation, that generates channel input symbols along with encoder and decoder actions. We prove that the limit of this sequence exists. For a given block length N and probability of error, , we give an upper bound on the maximum achievable rate. Our upper and lower bounds coincide and hence yield the capacity for the case where the probability of initial state is positive for all states. Further, for stationary indecomposable channels without intersymbol interference (ISI), the capacity is given as the limit of normalized directed information between the input and output sequence, maximized over an appropriate set of causally conditioned distributions. As an important special case, we consider the framework of 'to feed or not to feed back' where either the encoder or the decoder takes binary actions, which determine whether current channel output will be fed back to the encoder, with a constraint on the fraction of channel outputs that are fed back. As another special case of our framework, we characterize the capacity of 'coding on the backward link' in FSCs, i.e. when the decoder sends limited-rate instantaneous coded noise-free feedback on the backward link. Finally, we propose an extension of the Blahut-Arimoto algorithm for evaluating the capacity when actions can be cost constrained, and demonstrate its application on a few examples.

show abstract

To feed or not to feed back

Asnani¹,

Permuter²,

Weissman³

2011

2011 IEEE International Symposium on Information Theory Proceedings

View full text Add to dashboard Cite

show abstract

To Feed or Not to Feedback

Asnani

Permuter

Weissman

2014

IEEE Trans. Inform. Theory

View full text Add to dashboard Cite

An Examination of Practical Granger Causality Inference

Liu

Bahadori

2013

Proceedings of the 2013 SIAM International Conference on Data Mining

View full text Add to dashboard Cite

Learning temporal causal structures among multiple time series is one of the major tasks in mining time series data. Granger causality is one of the most popular techniques in uncovering the temporal dependencies among time series; however it faces two main challenges: (i) the spurious effect of unobserved time series and (ii) the computational challenges in high dimensional settings. In this paper, we utilize the confounder path delays to find a subset of time series that via conditioning on them we are able to cancel out the spurious confounder effects. After study of consistency of different Granger causality techniques, we propose Copula-Granger and show that while it is consistent in high dimensions, it can efficiently capture non-linearity in the data. Extensive experiments on a synthetic and a social networking dataset confirm our theoretical results. IntroductionIn the era of data deluge, we are confronted with largescale time series data, i.e., a sequence observations of concerned variables over a period of time. For example, terabytes of neural activity time series data are produced to record the collective response of neurons to different stimuli; petabytes of climate and meteorological data, such as temperature, solar radiation, and precipitation, are collected over the years; and exabytes of social media contents are generated over time on the Internet. A major data mining task for time series data is to uncover the temporal causal relationship among the time series. For example, in the climatology, we want to identify the factors that impact the climate patterns of certain regions. In social networks, we are interested in identification of the patterns of influence among users and how topics activate or suppress each other. Developing effective and scalable data mining algorithms to uncover temporal dependency structures between time series and reveal insights from data has become a key problem in machine learning and data mining.There are two major challenges in discovering temporal causal relationship in large-scale data: (i) not all influential confounders are observed in the datasets and

show abstract

Directed information, causal estimation, and communication in continuous time

Cited by 3 publications

References 43 publications

To feed or not to feed back

To feed or not to feed back

To Feed or Not to Feedback

An Examination of Practical Granger Causality Inference

Contact Info

Product

Resources

About