Quantifying behavior is crucial for many applications in neuroscience. Videography provides easy methods for the observation and recording of animal behavior in diverse settings, yet extracting particular aspects of a behavior for further analysis can be highly time consuming. In motor control studies, humans or other animals are often marked with reflective markers to assist with computer-based tracking, but markers are intrusive, and the number and location of the markers must be determined a priori. Here we present an efficient method for markerless pose estimation based on transfer learning with deep neural networks that achieves excellent results with minimal training data. We demonstrate the versatility of this framework by tracking various body parts in multiple species across a broad collection of behaviors. Remarkably, even when only a small number of frames are labeled (~200), the algorithm achieves excellent tracking performance on test frames that is comparable to human accuracy.

Biological motor control is versatile and efficient. Muscles are flexible and undergo continuous changes requiring distributed adaptive control mechanisms. How proprioception solves this problem in the brain is unknown. Here we pursue a task-driven modeling approach that has provided important insights into other sensory systems. However, unlike for vision and audition where large annotated datasets of raw images or sound are readily available, data of relevant proprioceptive stimuli are not. We generated a large-scale dataset of human arm trajectories as the hand is tracing the alphabet in 3D space, then using a musculoskeletal model derived the spindle firing rates during these movements. We propose an action recognition task that allows training of hierarchical models to classify the character identity from the spindle firing patterns. Artificial neural networks could robustly solve this task, and the networks' units show directional movement tuning akin to neurons in the primate somatosensory cortex. The same architectures with random weights also show similar kinematic feature tuning but do not reproduce the diversity of preferred directional tuning nor do they have invariant tuning across 3D space. Taken together our model is the first to link tuning properties in the proprioceptive system to the behavioral level. Proprioception | Goal-driven modeling | Handwritten character recognition | Deep neural networks | Musculoskeletal models | Somatosensory cortex | S1 | Cuneate NucleusHighlights:• We provide a normative approach to derive neural tuning of proprioceptive features from behaviorallydefined objectives.

Objective: Closed-loop prosthesis interfaces, which combine electromyography (EMG)-based control with supplementary feedback, represent a promising direction for developing the next generation of bionic limbs. However, we still lack an understanding of how users utilize these interfaces and how to evaluate competing solutions. In this study, we used the framework of speed-accuracy trade-off functions (SAF) to understand, evaluate, and compare the performance of two closed-loop user-prosthesis interfaces. Approach: Ten able-bodied participants and an amputee performed a force-matching task in a functional box-and-block setup at three different speeds. All participants were subjected to both interfaces in a crossover study design with a one-week washout period. Importantly, both interfaces used (identical) direct proportional control but differed in the feedback provided to the participant (EMG feedback vs. Force feedback). Therefore, we estimated the SAFs afforded by the two interfaces and sought to understand how the participants planned and executed the task under the various conditions. Main results: We found that execution speed significantly influenced performance, and that EMG feedback afforded better overall performance, especially at medium speeds. Notably, we found that there was a difference in the SAF between the two interfaces, with EMG feedback enabling participants to attain higher accuracies faster than Force feedback. Furthermore, both interfaces enabled participants to develop flexible control policies, while EMG feedback also afforded participants the ability to generate smoother, more repeatable EMG commands. Significance: Overall, the results indicate that the performance of closed-loop prosthesis interfaces depends critically on the feedback approach and execution speed. This study showed that the SAF framework could be used to reveal the differences between feedback approaches, which might not have been detected if the assessment was performed at a single speed. Therefore, we argue that it is important to consider the speed-accuracy trade-offs to rigorously evaluate and compare user-prosthesis interfaces.

Objective: Supplemental sensory feedback for myoelectric prostheses can provide both psychosocial and 9 functional benefits during prosthesis control. However, the impact of feedback depends on multiple factors 10 and there is insufficient understanding about the fundamental role of such feedback in prosthesis use. The 11 framework of human motor control enables us to systematically investigate the user-prosthesis control loop. 12 In this study, we explore how different task objectives such as speed and accuracy shape the control policy 13 developed by participants in a prosthesis force-matching task. 14 Approach: Participants were randomly assigned to two groups that both used identical EMG control 15 interface and prosthesis force feedback, through vibrotactile stimulation, to perform a prosthesis force-16 matching task. However, the groups received different task objectives specifying speed and accuracy 17 demands. We then investigated the control policies developed by the participants. To this end, we not only 18 evaluated how successful or fast participants were but also analyzed the behavioral strategies adopted by 19 the participants to obtain such performance gains. 20Main results: First, we observed that participants successfully integrated supplemental prosthesis force 21 feedback to develop both feedforward and feedback control policies, as demanded by the task objectives. 22We then observed that participants who first developed a (slow) feedback policy were quickly able to adapt 23 their policy to more stringent speed demands, by switching to a combined feedforward-feedback control 24 strategy. However, the participants who first developed a (fast) feedforward policy were not able to change 25 their control policy and adjust to greater accuracy demands. 26Significance: Overall, the results signify how the framework of human motor control can be applied to 27 study the role of feedback in user-prosthesis interaction. The results also reveal the utility of training 28 prosthesis users to integrate supplemental feedback into their state estimation by designing training 29 protocols that encourage the development of combined feedforward and feedback policy. 30

the posterior parietal cortex (ppc) and frontal motor areas comprise a cortical network supporting goal-directed behaviour, with functions including sensorimotor transformations and decision making. in primates, this network links performed and observed actions via mirror neurons, which fire both when individuals perform an action and when they observe the same action performed by a conspecific. Mirror neurons are believed to be important for social learning, but it is not known whether mirror-like neurons occur in similar networks in other social species, such as rodents, or if they can be measured in such models using paradigms where observers passively view a demonstrator. therefore, we imaged ca 2+ responses in PPC and secondary motor cortex (M2) while mice performed and observed pellet-reaching and wheel-running tasks, and found that cell populations in both areas robustly encoded several naturalistic behaviours. However, neural responses to the same set of observed actions were absent, although we verified that observer mice were attentive to performers and that PPC neurons responded reliably to visual cues. Statistical modelling also indicated that executed actions outperformed observed actions in predicting neural responses. these results raise the possibility that sensorimotor action recognition in rodents could take place outside of the parieto-frontal circuit, and underscore that detecting socially-driven neural coding depends critically on the species and behavioural paradigm used. A key function of any motor system is the rapid and flexible production of actions in response to external stimuli, including the behaviour of other individuals. Having robust representations of performed and observed behaviours has been hypothesized to add survival value in a number of species since it could facilitate optimal action selection, gaining access to food sources or avoiding predators 1. However, which neural circuits integrate performed and observed actions, and how, are not well understood. In different species of primates and songbirds, a striking manifestation of such interactions has been described in the form of mirror neurons. Mirror neurons, first characterized in pre-motor cortex 2,3 then PPC 4 in monkeys, and later reported in humans 5 and birds 6 , respond reliably both when an individual performs a specific action and when they observe the same action performed by a conspecific. Based on these properties they have been postulated to enable specific social functions ranging from selecting appropriate actions in response to observed behaviours 2 to understanding the intentions and imitating the actions of others 7,8. After years of investigation, however, it is still debated whether mirror cells are at the basis of action understanding or if their physiological properties can be better explained by simple, temporally contingent sensory-motor associations 9. Finding mechanistic resolutions to these questions would benefit tremendously if it were possible to access cellular networks underlying social le...