Decoding grasp and speech signals from the cortical grasp circuit in a tetraplegic human

Abstract

Tetraplegia from spinal cord injury leaves many patients paralyzed below the neck, leaving them unable to perform most activities of daily living. Brain-machine interfaces (BMIs) could give tetraplegic patients more independence by directly utilizing brain signals to control external devices such as robotic arms or hands. The cortical grasp network has been of particular interest because of its potential to facilitate the restoration of dexterous object manipulation. However, a network that involves such high-level cortical areas may also provide additional information, such as the encoding of speech. Towards understanding the role of different brain areas in the human cortical grasp network, neural activity related to motor intentions for grasping and performing speech was recorded in a tetraplegic patient in the supramarginal gyrus (SMG), the ventral premotor cortex (PMv), and the somatosensory cortex (S1). We found that in high-level brain areas SMG and PMv, grasps were well represented by firing rates of neuronal populations already at visual cue presentation. During motor imagery, grasps could be significantly decoded from all brain areas. At identical neuronal population sizes, SMG and PMv achieved similar highly-significant decoding abilities, demonstrating their potential for grasp BMIs. During speech, SMG encoded both spoken grasps and colors, in contrast to PMv and S1, which were not able to significantly decode speech.These findings suggest that grasp signals can robustly be decoded at a single unit level from the cortical grasping circuit in human. Data from PMv suggests a specialized role in grasping, while SMG’s role is broader and extends to speech. Together, these results indicate that brain signals from high-level areas of the human cortex can be exploited for a variety of different BMI applications.

Online internal speech decoding from single neurons in a human participant

“…We found evidence for phonetic representation during internal and vocalized speech. Adding to previous findings indicating grasp decoding in SMG (Wandelt et al 2022), we pose SMG as a multipurpose brain-machine interface area.…”

“…Impressive results in vocalized and attempted speech decoding and reconstruction have been achieved using these techniques (Angrick et al 2018; Herff et al 2019; Kellis et al 2010; Makin, Moses, and Chang 2020; Moses et al 2021). However, vocalized speech has also been decoded from small-scale microelectrode arrays located in the motor cortex (Stavisky et al 2019; Wilson et al 2020) and the supramarginal gyrus (SMG) (Wandelt et al 2022), demonstrating vocalized speech BMIs can be built using neural signals from localized regions of cortex.…”

“…Recently, (Makin, Moses, and Chang 2020) showed electrode grids over SMG contributed to vocalized speech decoding. Finally, vocalized grasps and color words were decodable from SMG from the same participant involved in this work (Wandelt et al 2022). These studies provide evidence for the possibility of an internal speech decoder from neural activity in SMG.…”

Speech decoding from a small set of spatially segregated minimally invasive intracranial EEG electrodes with a compact and interpretable neural network

“…There exist several reports exploiting intracortical activity recorded with Utah array like systems for speech prosthesis purposes [56, 54, 28]. These recordings give access to the activity of individual neurons but remain potentially harmful to the cortical tissue.…”

Direct Speech Reconstruction from Sensorimotor Brain Activity with Optimized Deep Learning Models

“…This work is primarily conducted in able-bodied human subjects who participate in speech production experiments while their brain activity is recorded, with the goal to infer spoken speech from the acquired brain signals. This setup is used as a testbed for development and validation of speech decoding frameworks prior to their use in real-world BCI applications with paralyzed individuals [29][30][31][32][33][34] . See [35][36][37][38][39][40][41][42][43][44][45] for reviews on BCIs for communication and speech.…”