Computing the trajectory mutual information between a point process and an analog stochastic process

Abstract

In a number of application areas such as neural coding there is interest in computing, from real data, the information flows between stochastic processes one of which is a point process. Of particular interest is the calculation of the trajectory (as opposed to marginal) mutual information between an observed point process which is influenced by an underlying but unobserved analog stochastic process i.e. a state. Using particle filtering we develop a model based trajectory mutual information calculation for apparently the first time.

Modulation Depth Estimation and Variable Selection in State-Space Models for Neural Interfaces

“…These quantities provide measures of instantaneous information flow between the two multivariate stochastic processes under consideration, namely x [ k ] and y [ k ]. Following the nomenclature used in recent work on mutual information estimation for point process data [36], we refer to this measure as the marginal SNR. We now consider the alternate problem of information flow between the entire trajectories of the random processes, as opposed to the instantaneous (marginal) measure.…”

Path mutual information for a class of biochemical reaction networks

“…Our goal is to quantify the information transmission between species A and B during this transient period. Note that the expressions (16), (17), (18) and (19) still apply, with the only difference that γ A = 0. In Fig.…”

“…A key advantage of information theoretic quantities is that they can be defined in very general terms such that they apply also to timespanning objects. Extensions of the mutual information to trajectories have been proposed [16,17,18,19] as well as the closely related transfer entropy [20,21,22]. However, the analysis of path-related information measures is mathematically more demanding, especially in the context of continuous-time jump processes such as frequently encountered in biochemical systems.…”

“…While the information theoretical foundation has been established for such processes [23,16,17], the explicit calculation of the path mutual information for concrete systems remains a challenge. Previous studies have addressed this problem either numerically using particle filters [17,19] or by approximating the original jump process by a continuous Gaussian process [18].…”

Mutual Information Flows in a Bivariate Point Process

“…An early reference is [4], but it only deals with instantaneous mutual information or marginal mutual information. However a study of dynamics needs measurement of mutual information between signal trajectories or histories -which we call trajectory mutual information [5]. Information flows are exactly relevant to the study of Granger causality in networks [6], [7], [8].…”