Investigating deep learning for fNIRS based BCI

Hennrich, Johannes; Herff, Christian; Heger, Dominic; Schultz, Tanja

doi:10.1109/embc.2015.7318984

Cited by 57 publications

(37 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…More recently, initial studies have successfully applied deep learning methods to brain data [25,26,27] and BCI applications [28,29,30,31,32,33]. Here, we show that densely-connected convolutional neural networks can be trained on limited training data to map ECoG dynamics directly to a speech spectrogram.…”

Section: Introductionmentioning

confidence: 76%

Speech Synthesis from ECoG using Densely Connected 3D Convolutional Neural Networks

Angrick

Mugler²,

Tate³

et al. 2018

Preprint

Self Cite

View full text Add to dashboard Cite

Objective. Direct synthesis of speech from neural signals could provide a fast and natural way of communication to people with neurological diseases. Invasively-measured brain activity (electrocorticography; ECoG) supplies the necessary temporal and spatial resolution to decode fast and complex processes such as speech production. A number of impressive advances in speech decoding using neural signals have been achieved in recent years, but the complex dynamics are still not fully understood. However, it is unlikely that simple linear models can capture the relation between neural activity and continuous spoken speech. Approach. Here we show that deep neural networks can be used to map ECoG from speech production areas onto an intermediate representation of speech (logMel spectrogram). The proposed method uses a densely connected convolutional neural network topology which is well-suited to work with the small amount of data available from each participant. Main results. In a study with six participants, we achieved correlations up to r = 0.69 between the reconstructed and original logMel spectrograms. We transfered our prediction back into an audible waveform by applying a Wavenet vocoder. The vocoder was conditioned on logMel features that harnessed a much larger, pre-existing data corpus to provide the most natural acoustic output. Significance. To the best of our knowledge, this is the first time that high-quality speech has been reconstructed from neural recordings during speech production using deep neural networks.

show abstract

Section: Introductionmentioning

confidence: 76%

Speech Synthesis from ECoG using Densely Connected 3D Convolutional Neural Networks

Angrick

Mugler²,

Tate³

et al. 2018

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…In addition, artificial intelligence methods based on deep learning have demonstrated their potential in enhancing the performance of BCI systems (Cecotti and Graser, 2011;Chiarelli et al, 2018;Lawhern et al, 2018;Nicholas et al, 2018;Sakhavi et al, 2018). Even though some studies have reported the superiority of the deep learning-based approach compared to the conventional machine learning methods (Trakoolwilaiwan et al, 2018), there still exist controversies regarding the employment of these opinions (Hennrich et al, 2015). Since deep learning techniques generally depend on human factors, such as how well the deep learning model structure is designed, objective and thorough investigations of deep learning models that can enhance the performance of NIRS-BCIs are necessary.…”

Section: Efforts To Improve the Performance Of Nirs-bcis: Future Persmentioning

confidence: 99%

Performance Improvement of Near-Infrared Spectroscopy-Based Brain-Computer Interface Using Regularized Linear Discriminant Analysis Ensemble Classifier Based on Bootstrap Aggregating

Shin

2020

Front. Neurosci.

View full text Add to dashboard Cite

Ensemble classifiers have been proven to result in better classification accuracy than that of a single strong learner in many machine learning studies. Although many studies on electroencephalography-brain-computer interface (BCI) used ensemble classifiers to enhance the BCI performance, ensemble classifiers have hardly been employed for near-infrared spectroscopy (NIRS)-BCIs. In addition, since there has not been any systematic and comparative study, the efficacy of ensemble classifiers for NIRS-BCIs remains unknown. In this study, four NIRS-BCI datasets were employed to evaluate the efficacy of linear discriminant analysis ensemble classifiers based on the bootstrap aggregating. From the analysis results, significant (or marginally significant) increases in the bitrate as well as the classification accuracy were found for all four NIRS-BCI datasets employed in this study. Moreover, significant bitrate improvements were found in two of the four datasets.

show abstract

“…However, it is useful to obtain the results in real-time to provide feedback to the clinicians. The deep learning method allows fast retrieval of parameters and been used recently in the diffuse optical field, including functional near-infrared spectroscopy [12][13][14], spatial frequency domain imaging [15], fluorescence imaging and tomography [16][17][18]. Its application for DCS has yet to be established [19].…”

Section: Introductionmentioning

confidence: 99%

Deep learning model for ultrafast quantification of blood flow in diffuse correlation spectroscopy

Poon

Long²,

Sunar

2020

Preprint

View full text Add to dashboard Cite

INDEX TERMStissue microvasculature blood flow, diffuse correlation spectroscopy, deep learning, convolution neural networks. ABSTRACTDiffuse correlation spectroscopy (DCS) is increasingly used in optical imaging field to assess blood flow in humans due to its non-invasive, real-time characteristics and its ability to provide label-free, bedside monitoring of blood flow changes. Previous DCS studies have utilized a traditional curve fitting of the analytical or Monte Carlo models to extract the blood flow changes, which is computationally demanding and less accurate as the signal to noise decreases. Here, we present a deep learning model that eliminates this bottleneck by solving the inverse problem more than 2300% faster, with equivalent or improved accuracy compared to the nonlinear fitting with an analytical method. The proposed deep learning inverse model will enable real-time and accurate tissue blood flow quantification with the DCS technique.

show abstract

Investigating deep learning for fNIRS based BCI

Cited by 57 publications

References 15 publications

Speech Synthesis from ECoG using Densely Connected 3D Convolutional Neural Networks

Speech Synthesis from ECoG using Densely Connected 3D Convolutional Neural Networks

Performance Improvement of Near-Infrared Spectroscopy-Based Brain-Computer Interface Using Regularized Linear Discriminant Analysis Ensemble Classifier Based on Bootstrap Aggregating

Deep learning model for ultrafast quantification of blood flow in diffuse correlation spectroscopy

Contact Info

Product

Resources

About