BENDR: using transformers and a contrastive self-supervised learning task to learn from massive amounts of EEG data

Kostas, Demetres; Aroca-Ouellette, Stéphane; Rudzicz, Frank

doi:10.48550/arxiv.2101.12037

Cited by 3 publications

(8 citation statements)

References 36 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Semi-supervised learning methods use both labeled and unlabeled data, usually to learn the structure of the training data to become able to generate more (artificial) training points , that are used for conventional supervised learning in a second learning phase. Self-supervised learning (Jing and Tian 2019) is a similar approach that is used to learn the relevant structure in EEG data by first learning an unsupervised pretext task, after which the model is further trained on the target task with labeled data (Banville et al 2020, Kostas et al 2021. The remainder of this review will focus on supervised learning methods.…”

Section: Machine Learningmentioning

confidence: 99%

“…A DL method that is useful in biosignal decoding is transfer learning (Fahimi et al 2019, Kostas et al 2021. With Transfer Learning, a model is first trained on general data from which the model aims to learn the structure of the data.…”

Section: Deep Learningmentioning

confidence: 99%

“…There are still several methods that are yet to be applied to biosignal decoding. For example self-supervised learning was recently used (Kostas et al 2021) for biosignal decoding, and there are multiple advancements in transfer learning (Srinivas et al 2021) that could help in mitigating the small data issue, with some techniques even allowing transfer between two completely different settings (Lu et al 2021). By leveraging these advancements, the calibration of the control system to a new user and the required user training could be minimized thanks to the strong generalization power of DL.…”

Section: Future Perspectivesmentioning

confidence: 99%

“…Self-supervised learning (Jing and Tian 2019) is a similar approach which is currently gaining traction in the larger ML community. This technique was previously used to learn the relevant structure in EEG data by first learning an unsupervised pretext task, after which the model is further trained on the target task with labeled data (Banville et al 2020, Kostas et al 2021. The remainder of this review will focus on supervised learning methods.…”

Section: Machine Learningmentioning

confidence: 99%

See 3 more Smart Citations

Deep learning for biosignal control: insights from basic to real-time methods with recommendations

et al. 2022

View full text Add to dashboard Cite

Biosignal control is an interaction modality that allows users to interact with electronic devices by decoding the biological signals emanating from the movements or thoughts of the user. This manner of interaction with devices can enhance the sense of agency for users and enable persons suffering from a paralyzing condition to interact with everyday devices that would otherwise be challenging for them to use. It can also improve control of prosthetic devices and exoskeletons by making the interaction feel more natural and intuitive. However, with the current state of the art, several issues still need to be addressed to reliably decode user intent from biosignals and provide an improved user experience over other interaction modalities. One solution is to leverage advances in Deep Learning (DL) methods to provide more reliable decoding at the expense of added computational complexity. This scoping review introduces the basic concepts of DL and assists readers in deploying DL methods to a real-time control system that should operate under real-world conditions. The scope of this review covers any electronic device, but with an emphasis on robotic devices, as this is the most active area of research in biosignal control. We review the literature pertaining to the implementation and evaluation of control systems that incorporate DL to identify the main gaps and issues in the field, and formulate suggestions on how to mitigate them. Additionally, we formulate guidelines on the best approach to designing, implementing and evaluating research prototypes that use DL in their biosignal control systems.

show abstract

Section: Machine Learningmentioning

confidence: 99%

Section: Deep Learningmentioning

confidence: 99%

Section: Future Perspectivesmentioning

confidence: 99%

Section: Machine Learningmentioning

confidence: 99%

See 2 more Smart Citations

Deep learning for biosignal control: insights from basic to real-time methods with recommendations

et al. 2022

View full text Add to dashboard Cite

show abstract

“…Kiyasseh et al (2020) created CLOCS, a family of contrastive learning methods on unlabeled cardiac physiological data for downstream tasks like better quantifying patient similarity for disease detection. Kostas et al (2021) created BENDR, which leverages transformers and contrastive self-supervised learning to better learn representations of electroencephalogram data. Moving to the realm of EHR data, Li et al (2019) built a framework that enhanced predictive performance for common diseases across multiple sites without the need to share data by leveraging Distributed Noise Contrastive Estimation.…”

Section: Related Workmentioning

confidence: 99%

Bootstrapping Your Own Positive Sample: Contrastive Learning With Electronic Health Record Data

Wanyan,

Zhang,

Ding

et al. 2021

Preprint

View full text Add to dashboard Cite

Electronic Health Record (EHR) data has been of tremendous utility in Artificial Intelligence (AI) for healthcare such as predicting future clinical events. These tasks, however, often come with many challenges when using classical machine learning models due to a myriad of factors including class imbalance and data heterogeneity (i.e., the complex intra-class variances). To address some of these research gaps, this paper leverages the exciting contrastive learning framework and proposes a novel contrastive regularized clinical classification model. The contrastive loss is found to substantially augment EHR-based prediction: it effectively characterizes the similar/dissimilar patterns (by its "push-andpull" form), meanwhile mitigating the highly skewed class distribution by learning more balanced feature spaces (as also echoed by recent findings). In particular, when naively exporting the contrastive learning to the EHR data, one hurdle is in generating positive samples, since EHR data is not as amendable to data augmentation as image data. To this end, we have introduced two unique positive sampling strategies specifically tailored for EHR data: a feature-based positive sampling that exploits the feature space neighborhood structure to reinforce the feature learning; and an attribute-based positive sampling that incorporates pre-generated patient similarity metrics to define the sample proximity. Both sampling approaches are designed with an awareness of unique high intra-class variance in EHR data. Our overall framework yields highly competitive experimental results in predicting the mortality risk on real-world COVID-19 EHR data with a total of 5,712 patients admitted to a large, urban health system. Specifically, our method reaches a

show abstract

Learning neural decoders without labels using multiple data streams

2022

View full text Add to dashboard Cite

Objective: Recent advances in neural decoding have accelerated the development of brain-computer interfaces aimed at assisting users with everyday tasks such as speaking, walking, and manipulating objects. However, current approaches for training neural decoders commonly require large quantities of labeled data, which can be laborious or infeasible to obtain in real-world settings. Alternatively, self-supervised models that share self-generated pseudo-labels between two data streams have shown exceptional performance on unlabeled audio and video data, but it remains unclear how well they extend to neural decoding. Approach: We learn neural decoders without labels by leveraging multiple simultaneously recorded data streams, including neural, kinematic, and physiological signals. Speciﬁcally, we apply cross-modal, self-supervised deep clustering to train decoders that can classify movements from brain recordings. After training, we then isolate the decoders for each input data stream and compare the accuracy of decoders trained using cross-modal deep clustering against supervised and unimodal, self-supervised models. Main results: We find that sharing pseudo-labels between two data streams during training substantially increases decoding performance compared to unimodal, self-supervised models, with accuracies approaching those of supervised decoders trained on labeled data. Next, we extend cross-modal decoder training to three or more modalities, achieving state-of-the-art neural decoding accuracy that matches or slightly exceeds the performance of supervised models. Signiﬁcance: We demonstrate that cross-modal, self-supervised decoding can be applied to train neural decoders when few or no labels are available and extend the cross-modal framework to share information among three or more data streams, further improving self-supervised training.

show abstract

BENDR: using transformers and a contrastive self-supervised learning task to learn from massive amounts of EEG data

Cited by 3 publications

References 36 publications

Deep learning for biosignal control: insights from basic to real-time methods with recommendations

Deep learning for biosignal control: insights from basic to real-time methods with recommendations

Bootstrapping Your Own Positive Sample: Contrastive Learning With Electronic Health Record Data

Learning neural decoders without labels using multiple data streams

Contact Info

Product

Resources

About