Wiring minimization of deep neural networks reveal conditions in which multiple visuotopic areas emerge

Obeid, Dina; Konkle, Talia

doi:10.1167/jov.21.9.2135

Cited by 2 publications

(1 citation statement)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Once the progenitor set is saturated, the most-strongly innervated set becomes the progenitor set, and the connection placement process repeats, until all cells are saturated or until cells in the target pool have run out (pseudocode and further detail in the Methods ). The process is local because growth at any point is focused on a subset of cells (growth does not happen globally); it is greedy because it proceeds through placement of the shortest available connections at the current moment and from the current set, rather than performing absolute wiring length minimization on the global circuit (as considered in earlier works [72, 54, 73, 74]).…”

Section: Resultsmentioning

confidence: 99%

Self-organized emergence of modularity, hierarchy, and mirror reversals from competitive synaptic growth in a developmental model of the visual pathway

Chandra,

Khona,

Konkle

et al. 2024

Preprint

Self Cite

View full text Add to dashboard Cite

A hallmark of the primate visual system is its architectural organization consisting of multiple distinct (modular) areas that connect hierarchically. These areas exhibit specific spatial organization on the cortical sheet, with primary visual cortex at the center and subsequent regions in the hierarchy encircling the earlier one, and detailed topological organization, with retinotopy in each area but striking mirror reversals across area boundaries. The developmental rules that drive the simultaneous formation of these architectural, spatial, and topographic aspects of organization are unknown. Here we demonstrate that a simple synaptic growth rule driven by spontaneous activity and heterosynaptic competition generates a detailed connectome of the visual pathway, with emergence of all three types of organization. We identify a theoretical principle --- local greedy wiring minimization via spontaneous drive (GWM-S) --- implemented by the mechanism, and use this insight to propose biologically distinct growth rules that predict similar endpoints but testably distinguishable developmental trajectories. The same rules predict how input geometry and cortical geometry together drive emergence of hierarchical, convolution-like, spatially and topographically organized sensory processing pathways for different modalities and species, providing a possible explanation for the observed pluripotency of cortical structure formation. We find that the few parameters governing structure emergence in the growth rule constitute simple knobs for rich control, that could (potentially genetically) encode a projection neuron-like connectivity patterns and interneuron-like ones. In all, the presented rules provide a parsimonious mechanistic model for the organization of sensory cortical hierarchies even without detailed genetic cues for features like map reversal, and provide numerous predictions for experiment during normal and perturbed development.

show abstract

Section: Resultsmentioning

confidence: 99%

Self-organized emergence of modularity, hierarchy, and mirror reversals from competitive synaptic growth in a developmental model of the visual pathway

Chandra,

Khona,

Konkle

et al. 2024

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

Winning the Lottery With Neural Connectivity Constraints: Faster Learning Across Cognitive Tasks With Spatially Constrained Sparse RNNs

Khona,

Chandra,

et al. 2023

Neural Computation

View full text Add to dashboard Cite

Recurrent neural networks (RNNs) are often used to model circuits in the brain and can solve a variety of difficult computational problems requiring memory, error correction, or selection (Hopfield, 1982; Maass et al., 2002; Maass, 2011). However, fully connected RNNs contrast structurally with their biological counterparts, which are extremely sparse (about 0.1%). Motivated by the neocortex, where neural connectivity is constrained by physical distance along cortical sheets and other synaptic wiring costs, we introduce locality masked RNNs (LM-RNNs) that use task-agnostic predetermined graphs with sparsity as low as 4%. We study LM-RNNs in a multitask learning setting relevant to cognitive systems neuroscience with a commonly used set of tasks, 20-Cog-tasks (Yang et al., 2019). We show through reductio ad absurdum that 20-Cog-tasks can be solved by a small pool of separated autapses that we can mechanistically analyze and understand. Thus, these tasks fall short of the goal of inducing complex recurrent dynamics and modular structure in RNNs. We next contribute a new cognitive multitask battery, Mod-Cog, consisting of up to 132 tasks that expands by about seven-fold the number of tasks and task complexity of 20-Cog-tasks. Importantly, while autapses can solve the simple 20-Cog-tasks, the expanded task set requires richer neural architectures and continuous attractor dynamics. On these tasks, we show that LM-RNNs with an optimal sparsity result in faster training and better data efficiency than fully connected networks.

show abstract

Wiring minimization of deep neural networks reveal conditions in which multiple visuotopic areas emerge

Cited by 2 publications

References 0 publications

Self-organized emergence of modularity, hierarchy, and mirror reversals from competitive synaptic growth in a developmental model of the visual pathway

Self-organized emergence of modularity, hierarchy, and mirror reversals from competitive synaptic growth in a developmental model of the visual pathway

Winning the Lottery With Neural Connectivity Constraints: Faster Learning Across Cognitive Tasks With Spatially Constrained Sparse RNNs

Contact Info

Product

Resources

About