Predictive visual motion extrapolation emerges spontaneously and without supervision at each layer of a hierarchical neural network with spike-timing-dependent plasticity

Abstract

The fact that the transmission and processing of visual information in the brain takes time presents a problem for the accurate real-time localisation of a moving object. One way this problem might be solved is extrapolation: using an objects past trajectory to predict its location in the present moment. Here, we investigate how a simulated in silico layered neural network might implement such extrapolation mechanisms, and how the necessary neural circuits might develop. We allowed an unsupervised hierarchical network of velocity-tuned neurons to learn its connectivity through spike-timing dependent plasticity. We show that the temporal contingencies between the different neural populations that are activated by an object as it moves causes the receptive fields of higher-level neurons to shift in the direction opposite to their preferred direction of motion. The result is that neural populations spontaneously start to represent moving objects as being further along their trajectory than where they were physically detected. Due to the inherent delays of neural transmission, this effectively compensates for (part of) those delays by bringing the represented position of a moving object closer to its instantaneous position in the world. Finally, we show that this model accurately predicts the pattern of perceptual mislocalisation that arises when human observers are required to localise a moving object relative to a flashed static object (the flash-lag effect).

Learning Spatiotemporal Properties of Hippocampal Place Cells

“…Though the asymmetric connectivity in the study by Tsodyks et al (1996) was manually set, this model is supported by experimental evidence that shows that hippocampal place fields are experience-dependent and can become asymmetric during route following (Mehta et al, 1997). In addition, this asymmetry can be generated by Hebbian plasticity, as supported by modelling studies that demonstrate how such asymmetric connectivity can be learnt via long-term potentiation/long-term depression (Mehta et al, 2000) or spike-timing-dependent plasticity (Burkitt and Hogendoorn, 2021) in an environment where the motion in the same direction is repeated many times. Apart from Hebbian plasticity, the asymmetric connectivity of the network may also arise from other sources: (1) it may arise during the developmental process, as supported by experimental studies which demonstrate that hippocampal place cells fire in sequences even before the exploration of a novel environment (Dragoi and Tonegawa, 2011); (2) it may be preconfigured as suggested by Mizuseki and Buzsáki (2013).…”

Resolving visual motion through perceptual gaps

“…likely to be the driving mechanism behind the flash-lag illusion [57] and explicit models have related extrapolation to the effect [9,102,103]. In particular, neural populations in an artificial neural network learn to extrapolate without supervision through spike-timing-dependent plasticity [9]. Comparing the extrapolation shift observed in the neural network with the magnitude of the flash-lag effect observed in humans [104] shows that the degree of extrapolation is highly similar (Figure 5A).…”

“…For example, advances in analysis methods for time-resolved neuroimaging methods such as electroencephalography (EEG) and magnetoencephalography (MEG) enable a deeper understanding of inherently dynamic processes [6] and have recently been employed to examine processes such as motion extrapolation [7,8]. In addition, modelling techniques have been used to examine how these processes could be instantiated in the brain [9].…”

“…Redrawn, with permission, from [87] (see Figure 3). likely to be the driving mechanism behind the flash-lag illusion [57] and explicit models have related extrapolation to the effect [9,102,103]. In particular, neural populations in an artificial neural network learn to extrapolate without supervision through spike-timing-dependent plasticity [9].…”

“…When correlating the FLE lag with the RF shift, there is a near-perfect correlation, suggesting that the mechanism of extrapolation occurs artificially in a similar way as behaviorally. Redrawn, with permission, from [9] (see Figures 7 and 8). (B) In another study [112], neural responses were recorded intracranially from the visual cortex while patients viewed images of various things such as faces, houses, and objects.…”