Contrasting action and posture coding with hierarchical deep neural network models of proprioception

Abstract

Biological motor control is versatile and efficient. Muscles are flexible and undergo continuous changes requiring distributed adaptive control mechanisms. How proprioception solves this problem in the brain is unknown. Here we pursue a task-driven modeling approach that has provided important insights into other sensory systems. However, unlike for vision and audition where large annotated datasets of raw images or sound are readily available, data of relevant proprioceptive stimuli are not. We generated a large-scale dataset of human arm trajectories as the hand is tracing the alphabet in 3D space, then using a musculoskeletal model derived the spindle firing rates during these movements. We propose an action recognition task that allows training of hierarchical models to classify the character identity from the spindle firing patterns. Artificial neural networks could robustly solve this task, and the networks' units show directional movement tuning akin to neurons in the primate somatosensory cortex. The same architectures with random weights also show similar kinematic feature tuning but do not reproduce the diversity of preferred directional tuning nor do they have invariant tuning across 3D space. Taken together our model is the first to link tuning properties in the proprioceptive system to the behavioral level. Proprioception | Goal-driven modeling | Handwritten character recognition | Deep neural networks | Musculoskeletal models | Somatosensory cortex | S1 | Cuneate NucleusHighlights:• We provide a normative approach to derive neural tuning of proprioceptive features from behaviorallydefined objectives.

Encoding of Limb State by Single Neurons in the Cuneate Nucleus of Awake Monkeys

“…The latter had an additional prominent lobe pointing away from the body, which was only weakly represented in the CN distribution. This bimodal PD distribution was predicted previously for both muscle spindles (Sandbrink et al, 2020) and neurons in primary motor cortex (Lillicrap & Scott, 2013). The discrepancy between simulated and actual CN PD distributions may be explained by a sampling bias introduced by the fixed depth of the recording electrodes.…”

Encoding of limb state by single neurons in the cuneate nucleus of awake monkeys

“…Although the major node of the CN PD distribution pointing toward the body closely matched that of the simulated distribution, the latter had an additional prominent lobe pointing away from the body, which was only weakly represented in the CN distribution. This bimodal PD distribution was predicted previously for both muscle spindles (26) and neurons in primary motor cortex (27). The discrepancy between simulated and actual CN PD distributions may be explained by a sampling bias introduced by the fixed depth of the recording electrodes.…”

CNN MouseNet: A biologically constrained convolutional neural network model for mouse visual cortex

“…Task-optimized deep networks show promise for brain modelling, because they are functionally sophisticated, and they often develop internal representations that overlap strongly with representations in the brain [8][9][10][11][12][13][14][15]. While deep network architectures are originally loosely inspired by the brain, there has been an extensive empirical exploration of the effects of architectural features in machine learning, in directions often independent from neuroscience.…”

“…Indeed, the activity in these networks often resembles activity recorded from areas of the primate visual system, from oriented Gabor-like features in early layers [4] to responses to curves and more complex geometries [5] and even functional, or representational, similarity at the population level [6,7]. Task-trained artificial neural networks have been shown to produce similar neural representations or develop predictive models of neural activity in visual [8][9][10], auditory [11], rodent whisker areas [12], and more [13][14][15]. Despite these successes and the clear power of CNNs to solve machine learning problems in the visual domain, among others [4,16], they are not structural or architectural analogues for the underlying biological circuits.…”