The functional role of oscillatory dynamics in neocortical circuits: a computational perspective

Abstract

Biological neuronal networks have the propensity to oscillate. However, it is unclear whether these oscillations are a mere byproduct of neuronal interactions or serve computational purposes. Therefore, we implemented hallmark features of the cerebral cortex in recurrent neuronal networks (RNNs) simulated in silico and examined their performance on common pattern recognition tasks after training with a gradient-based learning rule. We find that by configuring network nodes as damped harmonic oscillators (DHOs), performance is substantially improved over non-oscillating architectures and that the introduction of heterogeneous nodes, conduction delays, and network modularity further improved performance. We furthermore provide a proof of concept of how unsupervised Hebbian learning can work in such networks. Analyses of network activity illustrate how the nonlinear dynamics of coupled DHOs drive performance, and provide plausible a posteriori explanations for a number of physiological phenomena whose function so far has been elusive.

Oscillations in an Artificial Neural Network Convert Competing Inputs into a Temporal Code

“…Oscillatory activity in the theta, alpha, and gamma-band has been repeatedly shown to support learning and memory [for review see [33, 45]]. In line with this, Effenberger, Carvalho, Dubinin, and Singer [31] have shown that an RNN whose nodes reflect damped oscillators outperform the performance of non-oscillatory RNNs when classifying the sequential MNIST data set. For future extensions of the presented algorithm, it would be interesting to explore if and how biologically plausible dynamics could be used to support the training process.…”

“…For instance, in the form of neural ODEs [20], liquid-time constant neural networks [47,48], and RNNs consisting of damped oscillators [31]. These networks had great success in learning long-range dependencies in time series data [20,47,48], sequences of images [80], and image classification when the pixels of the input are transformed into a time series (sequential MNIST) [31,47]. The goal of our work, in comparison, was to convert spatially presented inputs into a time series.…”

“…Originally inspired by the receptive fields of neurons in visual cortex, the hierarchically organised representations emerging in these networks have been repeatedly shown to map on to those identified from human MEG and fMRI recordings of the visual ventral stream [24, 25, 42] and intracranial recordings from the non-human primate brain [73, 85, 100, 122, 121]. Despite the parallels between CNNs and the primate visual system, there are only few examples of ANNs for computer vision that have drawn inspiration from the temporal dynamics of cortical activity [31, 111]. For instance, alpha oscillations (8-12 Hz) dominate electrophysiological recordings from the human occipital lobe [1, 11], however, their computational benefit has so far not been explicitly explored in ANNs.…”

“…This work serves as a proof of principle to demonstrate the basic principles and analyse the ensuing dynamics in the network. Our model relates to previous work investigating emergent dynamical and oscillatory properties in systems for information storage [50] and Recurrent Neural Networks (RNNs) for image classification [31] and sequence learning [80]. In comparison to these works, we tune the dynamics of the system such that the network is able to segment the representations of competing stimuli along the phase of ongoing oscillations in the hidden units.…”

Joint encoding of stimulus and decision in monkey primary visual cortex

“…Recurrently coupled networks process signals in a highly parallel manner and represent computational results in complex dynamical landscapes to which all nodes of the network contribute continuously. Any local signal spreads immediately over the whole network ( Effenberger et al. 2022 ).…”