Clustering of Neural Activity: A Design Principle for Population Codes

Abstract

We propose that correlations among neurons are generically strong enough to organize neural activity patterns into a discrete set of clusters, which can each be viewed as a population codeword. Our reasoning starts with the analysis of retinal ganglion cell data using maximum entropy models, showing that the population is robustly in a frustrated, marginally sub-critical, or glassy, state. This leads to an argument that neural populations in many other brain areas might share this structure. Next, we use latent variable models to show that this glassy state possesses well-defined clusters of neural activity. Clusters have three appealing properties: (i) clusters exhibit error correction, i.e., they are reproducibly elicited by the same stimulus despite variability at the level of constituent neurons; (ii) clusters encode qualitatively different visual features than their constituent neurons; and (iii) clusters can be learned by downstream neural circuits in an unsupervised fashion. We hypothesize that these properties give rise to a "learnable" neural code which the cortical hierarchy uses to extract increasingly complex features without supervision or reinforcement.

A sparse set of spikes corresponding to reliable correlations is highly informative of visual stimulus on single trials

“…This type of coding scheme, where activity pattern of different structure have similar meaning has been termed a ‘semantic code’ 48,49 . In addition to their flexibility and learnability 50 , a semantic coding scheme allows extraction of meaning from sparse firing, as we found here. In turn, sparse coding has been hypothesized to increase the coding capacity of the network and to be metabolically efficient 51,52 .…”

A topological perspective on the dual nature of the neural state space and the correlation structure

“…Commonly in large-scale recordings, we find groups of neurons with high inter-correlation. These clusters are often referred to as neural ensembles and are presumed to solve specific computational tasks [22–24]. Studying the pairwise correlations of afformentioned orientation-tuned neurons in macaque monkeys, we again observe an underlying circular feature (Fig.…”

Long- and short-term history effects in a spiking network model of statistical learning

“…While theoretical work has provided insights into how individual neural ensembles may be formed (for a review see 26 ), it is an open question how to learn and compute in networks consisting of multiple ensembles. In general, a clustered code has interesting properties, such as robust error correction, that make it a candidate to underlie computations in the brain 41 – 43 . It has also been shown to enhance reinforcement learning in a recent study 44 .…”