Entropy and information in neural spike trains: Progress on the sampling problem

Nemenman, Ilya; Bialek, William; Steveninck, Rob de Ruyter van

doi:10.1103/physreve.69.056111

Cited by 243 publications

(353 citation statements)

References 27 publications

Supporting

Mentioning

333

Contrasting

Unclassified

Order By: Relevance

“…In this paper we present an information theoretic analysis of the effect of this synchrony on neural coding of direction and contrast, making use of a modified version of the method of information components Pola et al, 2003). We report here on a subset of 102 pairs (for the orientation analysis) and 71 pairs (for the contrast analysis) of the original Kohn and Smith (2005) data set, chosen to satisfy convergence criteria for the entropy estimation procedure that we used (Nemenman et al, 2004).…”

Section: Resultsmentioning

confidence: 99%

“…In doing so, it was crucial to use an effective sampling procedure (Nemenman et al, 2004) to avoid the results being contaminated by residual bias. The sampling approach we took is described in detail in supplemental material A (available at www.jneurosci.org).…”

Section: Discussionmentioning

confidence: 99%

“…That is, the stimulus-dependent correlation component is not made up entirely of entropies, and thus advanced entropy estimation techniques such as that by I. Nemenman, F. Shafee, and W. Bialek (NSB) (Nemenman et al, 2002(Nemenman et al, , 2004) cannot be applied. Moreover, can be biased substantially, which renders its calculation by previously available procedures difficult for all except very low-dimensional problems.…”

Section: Methodsmentioning

confidence: 99%

“…Although we do not see this as giving rise to problems of interpretation (but see Schneidman et al, 2003), it does mean that advanced methods for entropy estimation cannot be used. Thus we have rewritten the correlational components as approximate entropies (i.e., an entropy plus additional terms that are very small and relatively immune to bias), allowing us to use a recently developed technique for minimizing the bias associated with estimating entropy from a limited number of experimental samples (Nemenman et al, 2004). By using a Bayesian prior to generate a nearly uniform distribution of entropies, we thus correct for sample size-dependent bias at its source and avoid potential artifacts that may occur when sampling is insufficient, as is typically the case in neurophysiological experiments.…”

Section: Methodsmentioning

confidence: 99%

See 3 more Smart Citations

The Role of Correlations in Direction and Contrast Coding in the Primary Visual Cortex

Montani

Kohn²,

et al. 2007

J. Neurosci.

114

View full text Add to dashboard Cite

The spiking activity of nearby cortical neurons is not independent. Numerous studies have explored the importance of this correlated responsivity for visual coding and perception, often by comparing the information conveyed by pairs of simultaneously recorded neurons with the sum of information provided by the respective individual cells. Pairwise responses typically provide slightly more information so that encoding is weakly synergistic. The simple comparison between pairwise and summed individual responses conflates several forms of correlation, however, making it impossible to judge the relative importance of synchronous spiking, basic tuning properties, and stimulus-independent and stimulus-dependent correlation. We have applied an information theoretic approach to this question, using the responses of pairs of neurons to drifting sinusoidal gratings of different directions and contrasts that have been recorded in the primary visual cortex of anesthetized macaque monkeys. Our approach allows us to break down the information provided by pairs of neurons into a number of components. This analysis reveals that, although synchrony is prevalent and informative, the additional information it provides frequently is offset by the redundancy arising from the similar tuning properties of the two cells. Thus coding is approximately independent with weak synergy or redundancy arising, depending on the similarity in tuning and the temporal precision of the analysis. We suggest that this would allow cortical circuits to enjoy the stability provided by having similarly tuned neurons without suffering the penalty of redundancy, because the associated information transmission deficit is compensated for by stimulus-dependent synchrony.

show abstract

Section: Resultsmentioning

confidence: 99%

Section: Discussionmentioning

confidence: 99%

Section: Methodsmentioning

confidence: 99%

Section: Methodsmentioning

confidence: 99%

See 2 more Smart Citations

The Role of Correlations in Direction and Contrast Coding in the Primary Visual Cortex

Montani

Kohn²,

et al. 2007

J. Neurosci.

114

View full text Add to dashboard Cite

show abstract

“…Here the underlying idea, which is that one can assess the role of correlations by building decoders that ignore them, will be just as useful as the measure itself. This is because one does not actually have to calculate ⌬I [a difficult estimation problem, especially for population codes (Paninski, 2003(Paninski, , 2004Nemenman et al, 2004)], but instead one can build decoders that do and do not take some aspect of correlations into account. If taking correlations into account improves decoding accuracy, then correlations are important for decoding; otherwise, they are not.…”

Section: Discussionmentioning

confidence: 99%

Synergy, Redundancy, and Independence in Population Codes, Revisited

Latham¹,

Nirenberg²

2005

J. Neurosci.

206

239

View full text Add to dashboard Cite

Decoding the activity of a population of neurons is a fundamental problem in neuroscience. A key aspect of this problem is determining whether correlations in the activity, i.e., noise correlations, are important. If they are important, then the decoding problem is high dimensional: decoding algorithms must take the correlational structure in the activity into account. If they are not important, or if they play a minor role, then the decoding problem can be reduced to lower dimension and thus made more tractable. The issue of whether correlations are important has been a subject of heated debate. The debate centers around the validity of the measures used to address it. Here, we evaluate three of the most commonly used ones: synergy, ⌬I shuffled , and ⌬I. We show that synergy and ⌬I shuffled are confounded measures: they can be zero when correlations are clearly important for decoding and positive when they are not. In contrast, ⌬I is not confounded. It is zero only when correlations are not important for decoding and positive only when they are; that is, it is zero only when one can decode exactly as well using a decoder that ignores correlations as one can using a decoder that does not, and it is positive only when one cannot decode as well. Finally, we show that ⌬I has an information theoretic interpretation; it is an upper bound on the information lost when correlations are ignored.

show abstract

A statistical framework for neuroimaging data analysis based on mutual information estimated via a gaussian copula

Ince

Giordano

Kayser

et al. 2016

Human Brain Mapping

282

298

View full text Add to dashboard Cite

We begin by reviewing the statistical framework of information theory as applicable to neuroimaging data analysis. A major factor hindering wider adoption of this framework in neuroimaging is the difficulty of estimating information theoretic quantities in practice. We present a novel estimation technique that combines the statistical theory of copulas with the closed form solution for the entropy of Gaussian variables. This results in a general, computationally efficient, flexible, and robust multivariate statistical framework that provides effect sizes on a common meaningful scale, allows for unified treatment of discrete, continuous, unidimensional and multidimensional variables, and enables direct comparisons of representations from behavioral and brain responses across any recording modality. We validate the use of this estimate as a statistical test within a neuroimaging context, considering both discrete stimulus classes and continuous stimulus features. We also present examples of analyses facilitated by these developments, including application of multivariate analyses to MEG planar magnetic field gradients, and pairwise temporal interactions in evoked EEG responses. We show the benefit of considering the instantaneous temporal derivative together with the raw values of M/EEG signals as a multivariate response, how we can separately quantify modulations of amplitude and direction for vector quantities, and how we can measure the emergence of novel information over time in evoked responses. Open‐source Matlab and Python code implementing the new methods accompanies this article. Hum Brain Mapp 38:1541–1573, 2017. © 2016 Wiley Periodicals, Inc.

show abstract

Entropy and information in neural spike trains: Progress on the sampling problem

Cited by 243 publications

References 27 publications

The Role of Correlations in Direction and Contrast Coding in the Primary Visual Cortex

The Role of Correlations in Direction and Contrast Coding in the Primary Visual Cortex

Synergy, Redundancy, and Independence in Population Codes, Revisited

A statistical framework for neuroimaging data analysis based on mutual information estimated via a gaussian copula

Contact Info

Product

Resources

About