Anatomy of a bit: Information in a time series observation

James, Ryan G.; Ellison, Christopher J.; Crutchfield, James P.

doi:10.1063/1.3637494

Cited by 124 publications

(184 citation statements)

References 14 publications

Supporting

Mentioning

183

Contrasting

Order By: Relevance

“…We now describe a simple visual display [29] in which all the transmitted mutual information components appear, together with the residual output entropy. These displays are referred to as "spectra" because different colours are used for different components.…”

Section: Information Decomposition (Id) Spectramentioning

confidence: 99%

Partial and Entropic Information Decompositions of a Neuronal Modulatory Interaction

Kay

Ince

Dering

et al. 2017

Entropy

View full text Add to dashboard Cite

Information processing within neural systems often depends upon selective amplification of relevant signals and suppression of irrelevant signals. This has been shown many times by studies of contextual effects but there is as yet no consensus on how to interpret such studies. Some researchers interpret the effects of context as contributing to the selective receptive field (RF) input about which neurons transmit information. Others interpret context effects as affecting transmission of information about RF input without becoming part of the RF information transmitted. Here we use partial information decomposition (PID) and entropic information decomposition (EID) to study the properties of a form of modulation previously used in neurobiologically plausible neural nets. PID shows that this form of modulation can affect transmission of information in the RF input without the binary output transmitting any information unique to the modulator. EID produces similar decompositions, except that information unique to the modulator and the mechanistic shared component can be negative when modulating and modulated signals are correlated. Synergistic and source shared components were never negative in the conditions studied. Thus, both PID and EID show that modulatory inputs to a local processor can affect the transmission of information from other inputs. Contrary to what was previously assumed, this transmission can occur without the modulatory inputs becoming part of the information transmitted, as shown by the use of PID with the model we consider. Decompositions of psychophysical data from a visual contrast detection task with surrounding context suggest that a similar form of modulation may also occur in real neural systems.

show abstract

Section: Information Decomposition (Id) Spectramentioning

confidence: 99%

Partial and Entropic Information Decompositions of a Neuronal Modulatory Interaction

Kay

Ince

Dering

et al. 2017

Entropy

View full text Add to dashboard Cite

show abstract

“…We define continuous-time information anatomy [19] quantities as rates. As mentioned earlier, the present extends over an infinitesimal time.…”

Section: Differential Information Ratesmentioning

confidence: 99%

Informational and Causal Architecture of Continuous-time Renewal Processes

Marzen

Crutchfield

2017

J Stat Phys

Self Cite

View full text Add to dashboard Cite

We introduce the minimal maximally predictive models ( -machines) of processes generated by certain hidden semi-Markov models. Their causal states are either hybrid discrete-continuous or continuous random variables and causal-state transitions are described by partial differential equations. Closed-form expressions are given for statistical complexities, excess entropies, and differential information anatomy rates. We present a complete analysis of the -machines of continuous-time renewal processes and, then, extend this to processes generated by unifilar hidden semi-Markov models and semi-Markov models. Our information-theoretic analysis leads to new expressions for the entropy rate and the rates of related information measures for these very general continuous-time process classes.

show abstract

“…1, which shows how they partition the marginal entropy H(X t ), the uncertainty about a single observation in isolation; this partitioning is discussed in depth by James et al [9].…”

Section: Introductionmentioning

confidence: 92%

Predictive Information in Gaussian Processes with Application to Music Analysis

Abdallah

Plumbley

2013

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Abstract. We describe an information-theoretic approach to the analysis of sequential data, which emphasises the predictive aspects of perception, and the dynamic process of forming and modifying expectations about an unfolding stream of data, characterising these using a set of process information measures. After reviewing the theoretical foundations and the definition of the predictive information rate, we describe how this can be computed for Gaussian processes, including how the approach can be adpated to non-stationary processes, using an online Bayesian spectral estimation method to compute the Bayesian surprise. We finish with a sample analysis of a recording of Steve Reich's Drumming.

show abstract

Anatomy of a bit: Information in a time series observation

Cited by 124 publications

References 14 publications

Partial and Entropic Information Decompositions of a Neuronal Modulatory Interaction

Partial and Entropic Information Decompositions of a Neuronal Modulatory Interaction

Informational and Causal Architecture of Continuous-time Renewal Processes

Predictive Information in Gaussian Processes with Application to Music Analysis

Contact Info

Product

Resources

About