High-resolution neural recordings improve the accuracy of speech decoding

Abstract

Patients suffering from debilitating neurodegenerative diseases often lose the ability to communicate, detrimentally affecting their quality of life. One promising solution to restore communication is to decode signals directly from the brain to enable neural speech prostheses. However, decoding has been limited by coarse recordings, which inadequately capture the rich spatio-temporal structure of human brain signals. To resolve this limitation, we performed novel high-resolution, micro-electrocorticographic (µECoG) neural recordings during speech production. We obtained neural signals with 56× spatial resolution and 2.5× higher signal-to-noise ratio compared to standard invasive recordings. This increased signal quality nearly doubled decoding accuracy compared to standard intracranial signals. Accurate decoding was dependent on the high-spatial resolution of the neural interface. Further, non-linear decoding models designed to utilize enhanced spatio-temporal neural information produced better results than standard techniques. We show for the first time that micro-scale neural interfaces can enable high-quality speech decoding for neural speech prostheses.

Decoding articulatory and phonetic components of naturalistic continuous speech from the distributed language network

“…But only certain labels under each speech component were classified with high accuracy. Previous decoding studies using ECoG recordings have demonstrated higher accuracies for articulatory components [3,4] and phonemes [2,7], but most of these studies involved reading aloud simpler monosyllabic words with a consonant-vowelconsonant structure. These constrained word-sets limit the number of classes that can be decoded and may result in more consistent speech patterns which lead to higher decoding accuracies.…”

“…Speech brain-computer interfaces (speech-BCIs) are a potential tool to help these patients communicate with others by predicting their intended speech from neural activity. Electrocorticography (ECoG) and microelectrode arrays (MEAs) have been the most commonly used neural recording implants for BCI development, and have shown potential in decoding speech both in individuals with normal speech function [1][2][3][4][5][6][7] and in paralyzed patients with loss of speech function [8,9]. These neural implants have typically been placed in one cortical regionsensorimotor.…”

The Layer 7 Cortical Interface: A Scalable and Minimally Invasive Brain–Computer Interface Platform

“…Several groups have used advanced surface array techniques to correlate neural activity with motor function for the control of neural prostheses in paralyzed patients 7,84,85,[93][94][95][96] , to achieve speech decoding in anarthric patients 6,19,80,86,97,98 , or for other applications of high-resolution electrocorticography 72,80,[99][100][101][102][103][104][105][106][107][108][109] . Using the significantly increased resolution of the Layer 7 microelectrode array, we demonstrate neural decoding across multiple functional modalities, including gross and fine somatosensation, vision, and volitional motor function during awake, spontaneous, untrained behavior, as well as human speech, with a machine learning framework that suggests that work in the field to date has not yet fully capitalized on the electrical information present at the cortical surface.…”

“…It is not yet clear whether approaches involving softer penetrating electrodes offer a substantially different tradeoff 30,75 . For this reason, non-penetrating cortical surface microelectrodes represent a potentially attractive alternative 69,79,80 . In practice, electrocorticography (ECoG) has already facilitated capture of high quality signals for effective use in brain-computer interfaces in several applications, including motor and speech neural prostheses 7,19,32,34,72,[81][82][83][84][85][86][87] .…”

Acoustic inspired brain-to-sentence decoder for logosyllabic language

“…Most previous studies on language prosthesis use Electrocorticography (ECoG) electrodes 6,11,13,[19][20][21] or Utah array electrodes 4,10,22 to acquire cortical signals. However, some studies show that the subcortical signals have potential to enhance the effectiveness of decoder 23,24 .…”