Exploring and explaining properties of motion processing in biological brains using a neural network

Abstract

Visual motion perception underpins behaviors ranging from navigation to depth perception and grasping. Our limited access to biological systems constrains our understanding of how motion is processed within the brain. Here we explore properties of motion perception in biological systems by training a neural network to estimate the velocity of image sequences. The network recapitulates key characteristics of motion processing in biological brains, and we use our access to its structure to explore and understand motion (mis)perception. We find that the network captures the biological response to reverse-phi motion in terms of direction. We further find that it overestimates and underestimates the speed of slow and fast reverse-phi motion, respectively, because of the correlation between reverse-phi motion and the spatiotemporal receptive fields tuned to motion in opposite directions. Second, we find that the distribution of spatiotemporal tuning properties in the V1 and middle temporal (MT) layers of the network are similar to those observed in biological systems. We then show that, in comparison to MT units tuned to fast speeds, those tuned to slow speeds primarily receive input from V1 units tuned to high spatial frequency and low temporal frequency. Next, we find that there is a positive correlation between the pattern-motion and speed selectivity of MT units. Finally, we show that the network captures human underestimation of low coherence motion stimuli, and that this is due to pooling of noise and signal motion. These findings provide biologically plausible explanations for well-known phenomena and produce concrete predictions for future psychophysical and neurophysiological experiments.

Neural tuning instantiates prior expectations in the human visual system

“…Analogously, humans are superior on a range of visual tasks for stimuli that are oriented around cardinal orientations relative to oblique orientations 12 . Such biases in encoding and behaviour are even present in artificial intelligence systems trained on naturalistic movies 13 , 14 . Several computational accounts have attempted to unify the influence of environmental statistics on the properties of sensory neurons as well as perception 5 , 15 , 16 , but have been unable to address empirically how such encoding is implemented at the neural level.…”

Color and gloss constancy under diverse lighting environments

“…By analyzing the activation of units in the intermediate layers, we might be able to get a hint on specific image features that units in the network highly respond to (e.g., Flachot & Gegenfurtner 2018 , 2021 ). The complexity of the network and interpretability is in a trade-off relationship, and thus using a very shallow network would give a capacity to analyze characteristics of individual units in detail, allowing us to understand extracted features in a meaningful way ( Goncalves, & Welchman, 2017 ; Rideaux & Welchman, 2021 ).…”

Binocular vision supports the development of scene segmentation capabilities: Evidence from a deep learning model

“…Recently, this question has come under renewed focus with the rise of deep learning approaches in machine vision (cf., Lopez-Rubio, 2018;Majaj & Pelli, 2018;Richards et al, 2019). For many researchers in biological vision, deep learning networks provide an attractive and powerful way to conceive of the processes occurring in the mammalian visual system (Kriegeskorte, 2015;Rideaux & Welchman, 2020;Rideaux & Welchman, 2021;Srinath, Emonds, Wang, Lempel, Dunn-Weiss, Connor, & Nielsen, 2020). Like cells in the visual pathway, from retina to cortex, the filtering operations in DNNs make use of operations such as convolutions and max pooling, with some model architectures (e.g., "AlexNet"; Krizhevsky, Sutskever, & Hinton, 2017) exhibiting filter weights that bear similarity to the excitatory-inhibitory receptive field structures found in retinal ganglion cells, LGN and primary visual cortex.…”