Sparse synaptic connectivity is required for decorrelation and pattern separation in feedforward networks

Abstract

Pattern separation is a fundamental function of the brain. The divergent feedforward networks thought to underlie this computation are widespread, yet exhibit remarkably similar sparse synaptic connectivity. Marr-Albus theory postulates that such networks separate overlapping activity patterns by mapping them onto larger numbers of sparsely active neurons. But spatial correlations in synaptic input and those introduced by network connectivity are likely to compromise performance. To investigate the structural and functional determinants of pattern separation we built models of the cerebellar input layer with spatially correlated input patterns, and systematically varied their synaptic connectivity. Performance was quantified by the learning speed of a classifier trained on either the input or output patterns. Our results show that sparse synaptic connectivity is essential for separating spatially correlated input patterns over a wide range of network activity, and that expansion and correlations, rather than sparse activity, are the major determinants of pattern separation.

Expansion coding and computation in the cerebellum: 50 years after the Marr–Albus codon theory

“…From an anatomical viewpoint, this does not have to be the case; there are certainly many areas in brain with much higher connectivity. This suggests that there is some computational reason why such a small number is needed, and why larger numbers do not occur (Billings et al 2014;Cayco-Gajic et al 2017;Litwin-Kumar et al 2017;Sawtell, 2017). In the following sections we explore the computational consequences of this structure.…”

“…Expansion recoding. Pattern separation has been identified as an important justification for expansion coding (Billings et al 2014;D'Angelo, 2014;Cayco-Gajic et al 2017;Gilmer & Person, 2018;Cayco-Gajic & Silver, 2019). Discrete patterns are more likely to be linearly separable in a high-dimensional space, and therefore expansion coding enhances the ability of a linear Purkinje cell model to have flexible learned responses to differing inputs.…”

“…; Cayco‐Gajic et al . ). Theoretical analyses have examined the computational importance of the degree of connectivity (Billings et al .…”

“…; D'Angelo, ; Cayco‐Gajic et al . ; Gilmer & Person, ; Cayco‐Gajic & Silver, ). Discrete patterns are more likely to be linearly separable in a high‐dimensional space, and therefore expansion coding enhances the ability of a linear Purkinje cell model to have flexible learned responses to differing inputs.…”

“…Codon theory (Blomfield et al 1970;Marr, 1970) was named after the nucleotide triplets that comprise the genetic code, and its mechanism was based on earlier theoretical work of Brindley (1969). Re-coding of the input into discrete codon patterns has been proposed as a mechanism to permit flexible representation and rapid learning of non-linear functions and pattern discrimination through plasticity at the parallel fibre to Purkinje cell synapses (Blomfield et al 1970;Cayco-Gajic et al 2017). Theoretical analyses have examined the computational importance of the degree of connectivity (Billings et al 2014;Litwin-Kumar et al 2017).…”