Spatio-temporal deep learning for EEG-fNIRS brain computer interface

Ghonchi, Hamidreza; Fateh, Mansoor; Abolghasemi, Vahid; Ferdowsi, Saideh; Rezvani, Mohsen

doi:10.1109/embc44109.2020.9176183

Cited by 6 publications

(7 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…As data scarcity is one limiting factor of Deep Learning techniques (Ghonchi et al, 2020 ; Lu et al, 2020 ; Nagabushanam et al, 2020 ) the sliding window also augments the data by increasing the data’s sample size, as explained in Section “Proposed sliding window approach”.…”

Section: Discussionmentioning

confidence: 99%

See 1 more Smart Citation

Explainable artificial intelligence model to predict brain states from fNIRS signals

Shibu

Sreedharan

Arun

et al. 2023

Front. Hum. Neurosci.

View full text Add to dashboard Cite

Objective: Most Deep Learning (DL) methods for the classification of functional Near-Infrared Spectroscopy (fNIRS) signals do so without explaining which features contribute to the classification of a task or imagery. An explainable artificial intelligence (xAI) system that can decompose the Deep Learning mode’s output onto the input variables for fNIRS signals is described here.Approach: We propose an xAI-fNIRS system that consists of a classification module and an explanation module. The classification module consists of two separately trained sliding window-based classifiers, namely, (i) 1-D Convolutional Neural Network (CNN); and (ii) Long Short-Term Memory (LSTM). The explanation module uses SHAP (SHapley Additive exPlanations) to explain the CNN model’s output in terms of the model’s input.Main results: We observed that the classification module was able to classify two types of datasets: (a) Motor task (MT), acquired from three subjects; and (b) Motor imagery (MI), acquired from 29 subjects, with an accuracy of over 96% for both CNN and LSTM models. The explanation module was able to identify the channels contributing the most to the classification of MI or MT and therefore identify the channel locations and whether they correspond to oxy- or deoxy-hemoglobin levels in those locations.Significance: The xAI-fNIRS system can distinguish between the brain states related to overt and covert motor imagery from fNIRS signals with high classification accuracy and is able to explain the signal features that discriminate between the brain states of interest.

show abstract

Section: Discussionmentioning

confidence: 99%

“…In a recent study by Ghonchi et al ( 2020 ), spatiotemporal maps were extracted from fNIRS-EEG signals during a motor imagery task to classify the task conditions by a deep neural network. The authors were able to classify the brain states of motor imagery with an accuracy of 99%.…”

Section: Introductionmentioning

confidence: 99%

Explainable artificial intelligence model to predict brain states from fNIRS signals

Shibu

Sreedharan

Arun

et al. 2023

Front. Hum. Neurosci.

View full text Add to dashboard Cite

show abstract

“…The BCIs are applied to process this information along with a specific task, which can be used e.g., for detection purposes. In fact, the performance of the BCI systems strongly depends on the quality of provided information (in this case-brain activity related signal) [146,229]. The electroencephalography (EEG) using classic disc electrodes can be qualified as a non-invasive technique used to record brain activities from the scalp and to discern temporal information but it leaves out spatial information [152].…”

Section: Advanced Signal Processing Methods For Bci Systemsmentioning

confidence: 99%

“…A good alternative in order to overcome the above mentioned issues are hybrid BCI systems combining various brain imaging methods, such as EEG-fNIRS BCIs [91,145,146] or EEG-fMRI [136,147].…”

mentioning

confidence: 99%

Summary of over Fifty Years with Brain-Computer Interfaces—A Review

et al. 2021

View full text Add to dashboard Cite

Over the last few decades, the Brain-Computer Interfaces have been gradually making their way to the epicenter of scientific interest. Many scientists from all around the world have contributed to the state of the art in this scientific domain by developing numerous tools and methods for brain signal acquisition and processing. Such a spectacular progress would not be achievable without accompanying technological development to equip the researchers with the proper devices providing what is absolutely necessary for any kind of discovery as the core of every analysis: the data reflecting the brain activity. The common effort has resulted in pushing the whole domain to the point where the communication between a human being and the external world through BCI interfaces is no longer science fiction but nowadays reality. In this work we present the most relevant aspects of the BCIs and all the milestones that have been made over nearly 50-year history of this research domain. We mention people who were pioneers in this area as well as we highlight all the technological and methodological advances that have transformed something available and understandable by a very few into something that has a potential to be a breathtaking change for so many. Aiming to fully understand how the human brain works is a very ambitious goal and it will surely take time to succeed. However, even that fraction of what has already been determined is sufficient e.g., to allow impaired people to regain control on their lives and significantly improve its quality. The more is discovered in this domain, the more benefit for all of us this can potentially bring.

show abstract

“…In a motor imagery study, Ghonchi et al. 68 used fNIRS to augment the EEG data being collected. Three types of DL networks were used as classifiers, a CNN for its capability to extract special features, an LSTM for its ability to extract temporal features, and a recurrent CNN (RCNN) for its ability to extract both temporal and spatial features.…”

Section: Applications In Fnirsmentioning

confidence: 99%

Deep learning in fNIRS: a review

Eastmond

Subedi

et al. 2022

Neurophoton.

View full text Add to dashboard Cite

Significance: Optical neuroimaging has become a well-established clinical and research tool to monitor cortical activations in the human brain. It is notable that outcomes of functional nearinfrared spectroscopy (fNIRS) studies depend heavily on the data processing pipeline and classification model employed. Recently, deep learning (DL) methodologies have demonstrated fast and accurate performances in data processing and classification tasks across many biomedical fields.Aim: We aim to review the emerging DL applications in fNIRS studies.Approach: We first introduce some of the commonly used DL techniques. Then, the review summarizes current DL work in some of the most active areas of this field, including braincomputer interface, neuro-impairment diagnosis, and neuroscience discovery.Results: Of the 63 papers considered in this review, 32 report a comparative study of DL techniques to traditional machine learning techniques where 26 have been shown outperforming the latter in terms of the classification accuracy. In addition, eight studies also utilize DL to reduce the amount of preprocessing typically done with fNIRS data or increase the amount of data via data augmentation. Conclusions:The application of DL techniques to fNIRS studies has shown to mitigate many of the hurdles present in fNIRS studies such as lengthy data preprocessing or small sample sizes while achieving comparable or improved classification accuracy.

show abstract

Spatio-temporal deep learning for EEG-fNIRS brain computer interface

Cited by 6 publications

References 18 publications

Explainable artificial intelligence model to predict brain states from fNIRS signals

Explainable artificial intelligence model to predict brain states from fNIRS signals

Summary of over Fifty Years with Brain-Computer Interfaces—A Review

Deep learning in fNIRS: a review

Contact Info

Product

Resources

About