MotorNet: a Python toolbox for controlling differentiable biomechanical effectors with artificial neural networks

Abstract

Artificial neural networks (ANNs) are a powerful class of computational models for unravelling neural mechanisms of brain function. However, for neural control of movement, they currently must be integrated with software simulating biomechanical effectors, leading to limiting impracticalities: (1) researchers must rely on two different platforms and (2) biomechanical effectors are not generally differentiable, constraining researchers to reinforcement learning algorithms despite the existence and potential biological relevance of faster training methods. To address these limitations, we developed MotorNet, an open-source Python toolbox for creating arbitrarily complex, differentiable, and biomechanically realistic effectors that can be trained on user-defined motor tasks using ANNs. MotorNet is designed to meet several goals: ease of installation, ease of use, a high-level user-friendly API, and a modular architecture to allow for flexibility in model building. MotorNet requires no dependencies outside Python, making it easy to get started with. For instance, it allows training ANNs on typically used motor control models such as a two joint, six muscle, planar arm within minutes on a typical desktop computer. MotorNet is built on TensorFlow and therefore can implement any network architecture that is possible using the TensorFlow framework. Consequently, it will immediately benefit from advances in artificial intelligence through TensorFlow updates. Finally, it is open source, enabling users to create and share their own improvements, such as new effector and network architectures or custom task designs. MotorNet's focus on higher order model and task design will alleviate overhead cost to initiate computational projects for new researchers by providing a standalone, ready-to-go framework, and speed up efforts of established computational teams by enabling a focus on concepts and ideas over implementation.

Feedback control of recurrent dynamics constrains learning timescales during motor adaptation

“…Yet, many previous models of M1 have studied cortical dynamics in the absence of this feedback [15,18,67]. In this work, we reinterpret cortical trajectories as arising from an interaction between inputs and recurrent connectivity, recapitulating several previously reported features of M1 activity in feedback-driven networks (also see [12,13]). Moreover, this feedback signal may be flexibly updated to reshape effective cortical dynamics over short-term motor adaptation.…”

“…Moreover, the presence of feedback led us to consider the asymptotic properties of the joint RNN-cursor system, rather than the network dynamics alone. Future work can include additional dynamics of the controlled plant itself, whether that of a BCI effector or the musculo-skeletal system [12, 13]. Secondly, we examined the variability between different within-manifold decoder perturbations (WMPs), where all the new decoders are aligned to the intrinsic manifold.…”

“…While these theories have a rich history of interpreting and predicting behavioral dynamics, their application to understanding neural data has been more limited. Bridging this gap requires new network models incorporating feedback [11][12][13] as well as predictions about how neuronal interactions shape M1 activity structure for feedback control [14].…”

De novo motor learning creates structure in neural activity that shapes adaptation

“…6). We can expand on this work to examine more complex and realistic repertoires that include different behaviors like grasping and manipulation, along with models that consider arm kinematics 77,78 and dynamics 79 . Different actions have been shown to occupy different parts of neural space 13,15 , so different combinations of behaviors are likely to require different underlying manifolds, which would affect subsequent adaptation to perturbations on any given behavior.…”

μSim: A goal-driven framework for elucidating the neural control of movement through musculoskeletal modeling

“…However, these models lack flexible neural network implementations of the controller, and fail to generate neural level predictions. A recent approach called MotorNet [53] includes training controllers with differentiable biomechanical models; however, the correspondence of the resulting controllers to the MC or generalization to unseen conditions has yet not been established. Moreover, neural constraints that arise from the evolutionary standpoint cannot be implemented using these models, such as minimization of neural firing rates.…”