Machine learning algorithm for decoding multiple subthalamic spike trains for speech brain–machine interfaces

Abstract

Objective. The goal of this study is to decode the electrical activity of single neurons in the human subthalamic nucleus (STN) to infer the speech features that a person articulated, heard or imagined. We also aim to evaluate the amount of subthalamic neurons required for high accuracy decoding suitable for real-life speech brain-machine interfaces. Approach. We intraoperatively recorded single-neuron activity in the STN of 21 neurosurgical patients with Parkinson's disease undergoing implantation of deep brain stimulator (DBS) while patients produced, perceived or imagined the five monophthongal vowel sounds. Our decoder is based on machine learning algorithms that dynamically learn specific features of the speech-related firing patterns. Main results. In an extensive comparison of algorithms, our sparse decoder ("SpaDe"), based on sparse decomposition of the high dimensional neuronal feature space, outperformed the other algorithms in all three conditions: production, perception and imagery. For speech production, our algorithm, Spade, predicted all vowels correctly (accuracy: 100%; chance level: 20%). For perception accuracy was 96%, and for imagery: 88%. The accuracy of Spade showed a linear behavior in the amount of neurons for the perception data, and even faster for production or imagery. Significance. Our study demonstrates that the information encoded by single neurons in the STN about the production, perception and imagery of speech is suitable for high-accuracy decoding. It is therefore an important step towards brain-machine interfaces for restoration of speech faculties that bears an enormous potential to alleviate the suffering of completely paralyzed ("locked-in") patients and allow them to communicate again with their environment. Moreover, our research indicates how many subthalamic neurons may be necessary to achieve each level of decoding accuracy, which is of supreme importance for a neurosurgeon planning the implantation of a speech brain-machine interface.

Research on Tennis Motion Evaluation Method and System Based on Deep Learning

“…e automatic correction image exposure method is used to segment the input image first, and different regions in the image will be allocated to their respective region blocks according to the color gray threshold. After that, the original image is normalized and divided into 11 levels according to brightness [5]. e results of image segmentation are combined according to levels.…”

Machine learning decoding of single neurons in the thalamus for speech brain-machine interfaces

“…Our clinical procedure followed that of [41][42][43]53]. In brief, two microelectrodes (NeuroProbe-Sonus, Alpha Omega, Nof Hagalil, Israel) were temporarily implanted to locate the thalamus, based on neural activity patterns, using the Neuro Omega recording system (Alpha Omega, Nof Hagalil, Israel), advancing from 25 mm above the planned target along the selected trajectory, in 0.1-0.2 mm steps, until the exit from the thalamus (as detected by microelectrode recordings).…”

“…However, in contrast with the homogeneous encoding in the rAC/MOF and STG regions, some subthalamic neurons were sharplytuned, whereas others, broadly-tuned. We decoded the subthalamic neuronal activity to infer the five monophthongal vowel sounds [43]. Spade outperformed all other algorithms compared with and predicted the vowels with 100% accuracy for speech production, 96%, for perception, and 88%, for imagery.…”

Neuronal Encoding of Speech Features in the Human Thalamus in Parkinson's Disease and Essential Tremor Patients

“…The prevalent encoding by one-vowel sharp tuning also occurred in the ET patient population, but not in the PD one (Figure 6 middle column, middle and bottom rows) although, strictly speaking, the difference between the two groups did not reach statistical significance, presumably because of the overall low percentage of perception units (ET: 9.7% [7/72]; PD: 3.6% [2/55]; P = . 19).…”

“…Our study may also motivate research toward implantation of future speech brain-machine interfaces [55][56][57][58] in the human left Vim. Such interfaces will decode the neuronal activity and reconstruct speech, 19,59 for locked-in people, for example, patients with amyotrophic lateral sclerosis.…”