Generalized Bienenstock–Cooper–Munro rule for spiking neurons that maximizes information transmission

Abstract

Maximization of information transmission by a spiking-neuron model predicts changes of synaptic connections that depend on timing of pre-and postsynaptic spikes and on the postsynaptic membrane potential. Under the assumption of Poisson firing statistics, the synaptic update rule exhibits all of the features of the Bienenstock-Cooper-Munro rule, in particular, regimes of synaptic potentiation and depression separated by a sliding threshold. Moreover, the learning rule is also applicable to the more realistic case of neuron models with refractoriness, and is sensitive to correlations between input spikes, even in the absence of presynaptic rate modulation. The learning rule is found by maximizing the mutual information between presynaptic and postsynaptic spike trains under the constraint that the postsynaptic firing rate stays close to some target firing rate. An interpretation of the synaptic update rule in terms of homeostatic synaptic processes and spiketiming-dependent plasticity is discussed.computational neuroscience ͉ information theory ͉ learning ͉ spiking-neuron model ͉ synaptic plasticity T he efficacy of synaptic connections between neurons in the brain is not fixed, but it varies, depending on the firing frequency of presynaptic neurons (1, 2), the membrane potential of the postsynaptic neuron (3), spike timing (4-6), and intracellular parameters such as the calcium concentration; for a review see ref. 7. During the last decades, a large number of theoretical concepts and mathematical models have emerged that have helped to understand the functional consequences of synaptic modifications, in particular, long-term potentiation (LTP) and long-term depression (LTD) during development, learning, and memory; for reviews see (8)(9)(10). Apart from the work of Hebb (11), one of the most influential theoretical concepts has been the Bienenstock-Cooper-Munro (BCM) model originally developed to account for cortical organization and receptive field properties during development (12). The model predicted (i) regimes of both LTD and LTP, depending on the state of the postsynaptic neuron, and (ii) a sliding threshold that separates the two regimes. Both predictions i and ii have subsequently been confirmed experimentally (2,13,14).In this paper, we construct a bridge between the BCM model and a seemingly unconnected line of research in theoretical neuroscience centered around the concept of optimality.There are indications that several components of neural systems show close to optimal performance (15-17). Instead of looking at a specific implementation of synaptic changes, defined by a rule such as in the BCM model, we therefore ask what would be the optimal synaptic update rule so as to guarantee that a spiking neuron transmits as much information as possible? Information theoretic concepts have been used by several researchers because they allow to compare performance of neural systems with a fundamental theoretical limit (16, 17), but optimal synaptic update rules have so far been mostly restricted to a pure r...

Delay learning and polychronization for reservoir computing

“…As an alternative, various researchers have proposed different ways to exploit recent advances in neuroscience about synaptic plasticity [1], especially IP 2 [10,9] or STDP 3 [28,19], that is usually presented as the Hebb rule, revisited in the context of temporal coding. A current trend is to propose computational justifications for plasticity-based learning rules, in terms of entropy minimization [5] as well as log-likelihood [35] or mutual information maximization [8,46,7]. However, since STDP is a local unsupervised rule for adapting the weights of connections, such a synaptic plasticity is not efficient enough for controlling the behavior of an SNN in the context of a given task.…”

Differential Modulation of Motor Cortical Plasticity and Excitability in Early and Late Phases of Human Motor Learning

“…Theoretical concepts of synaptic modification as well as experimental work (Bienenstock et al, 1982;Kirkwood et al, 1996;Toyoizumi et al, 2005) suggest that the threshold for inducing either LTP or LTD at a synapse depends on the history of synaptic activity. According to this concept, previous activity reduces the probability for LTP-induction whereas it increases the probability for LTD-induction.…”

Perceptron learning rule derived from spike-frequency adaptation and spike-time-dependent plasticity

“…Such a scenario could also apply to cortex, where SFA and STDP coexist (30)(31)(32)(33) and where decay of feedforward connections could be prevented by delayed inputs arising from feedback loops via higher cortical areas (34,35). In computational learning theories, STDP has been linked to temporal difference learning (36) and to maximization of mutual information (37). Here, we extend this list of computational functions of STDP to include gradient-descent error minimization.…”