On Criticality in High-Dimensional Data

Saremi, Saeed; Sejnowski, Terrence J.

doi:10.1162/neco_a_00607

Cited by 9 publications

(11 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…log-probabilities) and hence a large specific heat. It has also been argued that the use of data sets which are too small might give rise to spuriously big specific heats [ 49 ]: while this could be true in principle, additional analyses e.g. in [ 7 ] show that their results are robust with respect to data set size, and our results are also valid even in the case of infinite data.…”

Section: Discussionsupporting

confidence: 55%

Signatures of criticality arise from random subsampling in simple population models

et al. 2017

View full text Add to dashboard Cite

The rise of large-scale recordings of neuronal activity has fueled the hope to gain new insights into the collective activity of neural ensembles. How can one link the statistics of neural population activity to underlying principles and theories? One attempt to interpret such data builds upon analogies to the behaviour of collective systems in statistical physics. Divergence of the specific heat—a measure of population statistics derived from thermodynamics—has been used to suggest that neural populations are optimized to operate at a “critical point”. However, these findings have been challenged by theoretical studies which have shown that common inputs can lead to diverging specific heat. Here, we connect “signatures of criticality”, and in particular the divergence of specific heat, back to statistics of neural population activity commonly studied in neural coding: firing rates and pairwise correlations. We show that the specific heat diverges whenever the average correlation strength does not depend on population size. This is necessarily true when data with correlations is randomly subsampled during the analysis process, irrespective of the detailed structure or origin of correlations. We also show how the characteristic shape of specific heat capacity curves depends on firing rates and correlations, using both analytically tractable models and numerical simulations of a canonical feed-forward population model. To analyze these simulations, we develop efficient methods for characterizing large-scale neural population activity with maximum entropy models. We find that, consistent with experimental findings, increases in firing rates and correlation directly lead to more pronounced signatures. Thus, previous reports of thermodynamical criticality in neural populations based on the analysis of specific heat can be explained by average firing rates and correlations, and are not indicative of an optimized coding strategy. We conclude that a reliable interpretation of statistical tests for theories of neural coding is possible only in reference to relevant ground-truth models.

show abstract

Section: Discussionsupporting

confidence: 55%

Signatures of criticality arise from random subsampling in simple population models

et al. 2017

View full text Add to dashboard Cite

show abstract

“…Indeed, several studies have suggested evidence of statistical criticality in neural data [ 32 , 33 , 34 ]. However, other studies call the significance of this into question [ 35 ], showing that these statistics can arise under very generic circumstances [ 24 ], might be inherited from the environment [ 36 ], and could even be a data-processing artefact [ 37 ] or due to insufficient sample sizes [ 38 ]. We hope to clarify some of this controversy by examining the emergence of statistical criticality in this in silico model of spiking population coding.…”

Section: Resultsmentioning

confidence: 99%

“…Here we show that statistical criticality emerges naturally in a model of stochastic spiking encoding, but only for models that are large enough to capture the stimulus distribution. Our use of a ground-truth model simulation ensures that these statistics are not an artefact of recording or data-processing [ 37 , 38 ]. A natural question, however, is whether these statistics arise from the statistics of natural images, which also exhibit Zipf’s law [ 40 ].…”

Section: Resultsmentioning

confidence: 99%

Optimal Encoding in Stochastic Latent-Variable Models

Rule

Sorbaro

Hennig

2020

Entropy

View full text Add to dashboard Cite

In this work we explore encoding strategies learned by statistical models of sensory coding in noisy spiking networks. Early stages of sensory communication in neural systems can be viewed as encoding channels in the information-theoretic sense. However, neural populations face constraints not commonly considered in communications theory. Using restricted Boltzmann machines as a model of sensory encoding, we find that networks with sufficient capacity learn to balance precision and noise-robustness in order to adaptively communicate stimuli with varying information content. Mirroring variability suppression observed in sensory systems, informative stimuli are encoded with high precision, at the cost of more variable responses to frequent, hence less informative stimuli. Curiously, we also find that statistical criticality in the neural population code emerges at model sizes where the input statistics are well captured. These phenomena have well-defined thermodynamic interpretations, and we discuss their connection to prevailing theories of coding and statistical criticality in neural populations.

show abstract

“…Therefore, on average, activation of a pixel in any of the bit planes above layer 6 (λ ≤ 6) will make the pixel value greater than the median. The median-thresholded images Θ can therefore be obtained by combining ℬ λ above layer 6 using the logical OR operator [9]: (4) We applied our analysis for layers ℬ λ to the ensemble Θ. As we will see, the medianthresholded images Θ are close to a percolation transition (Fig.…”

Section: Binary Representation Of Imagesmentioning

confidence: 95%

Correlated Percolation, Fractal Structures, and Scale-Invariant Distribution of Clusters in Natural Images

Saremi¹,

Sejnowski²

2016

IEEE Trans. Pattern Anal. Mach. Intell.

Self Cite

View full text Add to dashboard Cite

Natural images are scale invariant with structures at all length scales. We formulated a geometric view of scale invariance in natural images using percolation theory, which describes the behavior of connected clusters on graphs. We map images to the percolation model by defining clusters on a binary representation for images. We show that critical percolating structures emerge in natural images and study their scaling properties by identifying fractal dimensions and exponents for the scale-invariant distributions of clusters. This formulation leads to a method for identifying clusters in images from underlying structures as a starting point for image segmentation.

show abstract

On Criticality in High-Dimensional Data

Cited by 9 publications

References 15 publications

Signatures of criticality arise from random subsampling in simple population models

Signatures of criticality arise from random subsampling in simple population models

Optimal Encoding in Stochastic Latent-Variable Models

Correlated Percolation, Fractal Structures, and Scale-Invariant Distribution of Clusters in Natural Images

Contact Info

Product

Resources

About