Exploratory methods for high-performance EEG speech decoding

Abstract

State-of-the-art technologies in neural speech decoding utilize data collected from microwires or microarrays implanted directly into the cerebral cortex. Yet as a tool accessible only to individuals with implanted electrodes, speech decoding from devices of this nature is severely limited in its implementation, and cannot be considered a viable solution for widespread application. Speech decoding from non-invasive EEG signals can achieve relatively high accuracy (70-80%), but only from very small classification tasks, with more complex tasks typically yielding a limited (20-50%) classification accuracy. We propose a novel combination of technologies in which transcranial magnetic stimulation (TMS) is first applied to augment the neural signals of interest, producing a greater signal-to-noise ratio in the EEG data. Next, delay differential analysis (DDA) – a cutting-edge computational method based on nonlinear dynamics – is implemented to capture the widest range of information available in the neural signal, by incorporating both linear and nonlinear dynamics.

An open-access EEG dataset for speech decoding: Exploring the role of articulation and coarticulation

“…This paradigm improved subject performance in phoneme discrimination (in matched trials) by administering two closely spaced TMS pulses prior to phoneme presentation 54 . There is evidence that TMS may facilitate speech decoding when stimulation to the motor cortex target regions that control the muscles involved in the production of a specific phoneme category, and errors in decoding may be induced when stimulation occurs in an unrelated region of the motor cortex, prompting the perception of another phoneme category 51,54 . A mismatch between the phoneme type and stimulation site will introduce uncertainty into the decoding task, similar to when a phoneme is misheard in everyday speech.…”

“…We recommend that the larger dataset be used for model development and that the smaller dataset be used to validate the model in more stringent conditions. Our own tests have shown that, by means of DDA, we can successfully perform a classification analysis with both data sets 51 .…”

“…In addition to the limitations of EEG data (noise from within the signal), sounds are often heard inaccurately in the presence of environmental noise or when the participant is inattentive to stimuli 48–50 . We provide data from a control condition and two TMS conditions that facilitate or impede task performance, depending on the phoneme type and stimulation target 51 . Speech decoding papers typically publish one or more analyses conducted on a single dataset. This raises concerns about overfitting and how well the model might perform on additional data without significant modification 52 .…”

“…In addition to the limitations of EEG data (noise from within the signal), sounds are often heard inaccurately in the presence of environmental noise or when the participant is inattentive to stimuli 48–50 . We provide data from a control condition and two TMS conditions that facilitate or impede task performance, depending on the phoneme type and stimulation target 51 .…”

“…A wide variety of TMS paradigms have been tested with conflicting results, such that it may be better to observe the TEP in order to identify whether the paradigm was excitatory or inhibitory, or to consider the effect by means of an additional measure, such as a behavioral task (see 7C) 78 . The TMS paradigm used for collection of the dataset produced a facilitatory effect on performance in a phoneme perception task 51,54 .…”