High-resolution neural recordings improve the accuracy of speech decoding

Abstract

Patients suffering from debilitating neurodegenerative diseases often lose the ability to communicate, detrimentally affecting their quality of life. One solution to restore communication is to decode signals directly from the brain to enable neural speech prostheses. However, decoding has been limited by coarse neural recordings which inadequately capture the rich spatio-temporal structure of human brain signals. To resolve this limitation, we performed high-resolution, micro-electrocorticographic (µECoG) neural recordings during intra-operative speech production. We obtained neural signals with 57× higher spatial resolution and 48% higher signal-to-noise ratio compared to macro-ECoG and SEEG. This increased signal quality improved decoding by 35% compared to standard intracranial signals. Accurate decoding was dependent on the high-spatial resolution of the neural interface. Non-linear decoding models designed to utilize enhanced spatio-temporal neural information produced better results than linear techniques. We show that high-density µECoG can enable high-quality speech decoding for future neural speech prostheses.

Text and image generation from intracranial electroencephalography using an embedding space for text and images

“…In addition, although ridge regression is a common algorithm for decoding fMRI signals with visual stimuli to generate text [20,21] and images [18,28], other machine learning algorithms can improve the accuracy of inferring the CLIP vector from ECoGs. Some previous studies using ECoGs succeeded in improving the decoding accuracy using variational Bayesian decoding [29] and long short-term memory with/without a recurrent neural network [30][31][32]. Moreover, some signal feature extraction methods, such as dynamic mode decomposition [33] and convolutional neural networks [34,35], also improve the decoding accuracy.…”

A brain-to-text framework of decoding natural tonal sentences

“…A theoretical foundation of these studies is that there exist spatially distinct and distributed neural populations in the vSMC, representing different articulatory gestures corresponding to phoneme and pitch articulation. High-density ECoG recordings have proven to reliably cover the distributed network and dissociate these fine-grained neural coding 13,29 .…”

“…Examples include the Utah array 9,37 , stereoelectroencephalography (SEEG) 38 , and neuropixels 39,40 . When choosing among these methods, a crucial consideration involves striking a balance between obtaining high-resolution neural signals, such as investigating fine-grained spiking properties of multiple single units or microcircuits underlying speech production within a limited area of the cortex 9,40 , and achieving broad coverage of cortical networks, such as collecting neural signals across the entire vSMC, which depicts a comprehensive view of neurodynamics of the functional regions 21,22,29 . Future works remain to be done to investigate the decoding capabilities of natural tonal languages using signals of varying coverage and resolution scales.…”

Modular Phoneme Processing in Human Superior Temporal Gyrus

“…Conversely, ∼250μm diameter contacts at ∼1mm pitch would result in recordings from all modules, in most cases without significant crosstalk. Although these estimates are very rough, and assume that the modular organization exists in other association areas and are the same size, they suggest that finer sampling may reveal important clinical 16 and basic information, and provide the approximate spatial sampling necessary in language prostheses for detecting all functional units.…”