The Impact of Task Context on Predicting Finger Movements in a Brain-Machine Interface

Abstract

A key factor in the clinical translation of brain-machine interfaces (BMIs) for restoring hand motor function will be their robustness to changes in a task. With functional electrical stimulation (FES) for example, the patient's own hand will be used to produce a wide range of forces in otherwise similar movements. To investigate the impact of task changes on BMI performance, we trained two rhesus macaques to control a virtual hand with their physical hand while we added springs to each finger group (index or middle-ring-small) or altered their wrist posture. Using simultaneously recorded intracortical neural activity, finger positions, and electromyography, we found that predicting finger kinematics and finger-related muscle activations across contexts led to significant increases in prediction error, especially for muscle activations. However, with respect to online BMI control of the virtual hand, changing either training task context or the hand's physical context during online control had little effect on online performance. We explain this dichotomy by showing that the structure of neural population activity remained similar in new contexts, which could allow for fast adjustment online. Additionally, we found that neural activity shifted trajectories proportional to the required muscle activation in new contexts, possibly explaining biased kinematic predictions and suggesting a feature that could help predict different magnitude muscle activations while producing similar kinematics.

Artificial neural network for brain-machine interface consistently produces more naturalistic finger movements than linear methods

“…One non-human primate (NHP), Monkey N (NHP, Rhesus macaque, Macaca mulatta ) was implanted with Utah microelectrode arrays (Blackrock Neurotech, Salt Lake City, UT) in the hand area of the precentral gyrus, as described previously ( 16 , 32 , 61 , 62 ). Monkey N was implanted with two 64-channel arrays in the right hemisphere of motor cortex (precentral gyrus), (seen in Fig.…”

“…This means that positive velocities are flexion and negative velocities are extension. On some days we shifted the ‘context’ of the task either by adding springs to the manipulandum to increase the required flexion force or by rotating the wrist angle, both of which have previously been shown to modulate neural activity while maintaining similar kinematics ( 32 ). Applying these shifts results in four task ‘contexts’: the spring context, the wrist context , spring-wrist context (both simultaneously), and normal context (no shifts).…”

“…However, more recent work has suggested that the true relationship is more complicated: neural activity is driven at a population level by latent 'neural trajectories' through lower dimensional neural subspaces/manifolds, and these trajectories may have a highly nonlinear relationship to behavior (25)(26)(27)(28)(29)(30)(31). While linear decoders may be especially capable of predicting multi-DOF single-effector movements in constrained settings, recent studies have observed that linear decoders tested outside of their specific task may break down (29,32). As tasks become more sophisticated, the field had looked to nonlinear decoders as a potential solution for high-performance BMIs.…”

“…While the use of artificial neural network (ANN) decoders has significantly improved BMI performance in tightly controlled experimental settings, it is unclear how well they will generalize to real-world use cases. As neural activity has been shown to shift significantly between variations to a task, it’s possible that ANNs may be sacrificing robustness to achieve such high-performance ( 32 ). A recent study demonstrated a recurrent neural network being effectively used to control two cursors simultaneously in real time, but it also demonstrated that without proper training, models may overfit to training datasets and fail during closed-loop control ( 36 ).…”