Long-Term Mutual Training for the CYBATHLON BCI Race With a Tetraplegic Pilot: A Case Study on Inter-Session Transfer and Intra-Session Adaptation

Abstract

CYBATHLON is an international championship where people with severe physical disabilities compete with the aid of state-of-the-art assistive technology. In one of the disciplines, the BCI Race, tetraplegic pilots compete in a computer game race by controlling an avatar with a brain-computer interface (BCI). This competition offers a perfect opportunity for BCI researchers to study long-term training effects in potential end-users, and to evaluate BCI performance in a realistic environment. In this work, we describe the BCI system designed by the team Mirage91 for participation in the CYBATHLON BCI Series 2019, as well as in the CYBATHLON 2020 Global Edition. Furthermore, we present the BCI’s interface with the game and the main methodological strategies, along with a detailed evaluation of its performance over the course of the training period, which lasted 14 months. The developed system was a 4-class BCI relying on task-specific modulations of brain rhythms. We implemented inter-session transfer learning to reduce calibration time, and to reinforce the stability of the brain patterns. Additionally, in order to compensate for potential intra-session shifts in the features’ distribution, normalization parameters were continuously adapted in an unsupervised fashion. Across the aforementioned 14 months, we recorded 26 game-based training sessions. Between the first eight sessions, and the final eight sessions leading up to the CYBATHLON 2020 Global Edition, the runtimes significantly improved from 255 ± 23 s (mean ± std) to 225 ± 22 s, respectively. Moreover, we observed a significant increase in the classifier’s accuracy from 46 to 53%, driven by more distinguishable brain patterns. Compared to conventional single session, non-adaptive BCIs, the inter-session transfer learning and unsupervised intra-session adaptation techniques significantly improved the performance. This long-term study demonstrates that regular training helped the pilot to significantly increase the distance between task-specific patterns, which resulted in an improvement of performance, both with respect to class separability in the calibration data, and with respect to the game. Furthermore, it shows that our methodological approaches were beneficial in transferring the performance across sessions, and most importantly to the CYBATHLON competitions.

Neural correlates of user learning during long-term BCI training for the Cybathlon competition

“…The first attempt to investigate the evolution of the user’s brain activity concerned the analysis of the EEG features distribution used to create the decoder (i.e., the channels’ domain) since it directly reflects the ability of the user to control the BCI. In particular, the emergence of SMRs modulations associated with the mental tasks is the most commonly used index in the BCI literature to monitor longitudinal improvements [ 14 , 15 , 34 – 36 , 54 ]. Except for [ 34 ], none of these studies explicitly related the evolution of SMRs modulations to consistent and continuous changes of BCI performance.…”

“…6 b), indicating a more holistic phenomenon which is not limited to class-specific discriminant features. While an increase of the within-class distance is usually seen negatively in the literature—and a lot of effort is spent in the development of machine learning algorithms to minimize it [ 14 , 36 ]—recent literature promotes the idea that it could be linked with a positive user adaptation to the BCI system [ 20 , 35 ]. The user modifies his brain activity to produce brain signal modulations that match more closely those expected by the decoder [ 35 ].…”

“…Indeed, the other teams adopted a training strategy focused on frequent re-calibrations of their decoders [ 35 – 38 ]. Except for [ 36 ], all the other teams reported a significant worsening of their pilots’ performances during the competitions, explained by the presence of audience or, in general, conditions of stress. It is well-known that the psychophysiological state of the user has a negative effect on BCI operation [ 69 – 71 ].…”

“…In particular, works from [ 34 – 38 ] report on the experience in the BCI Race Discipline of the Cybathlon events (i.e., 2016 Cybathlon, 2019 Cybathlon BCI Series in Graz and the 2020 Cybathlon Global edition). While in [ 37 , 38 ] the importance of the training and the user’s engagement during closed-loop BCI operations has been clearly highlighted, in [ 35 , 36 ] the authors described the machine learning approaches for unsupervised adaptive re-calibration exploited by the teams to reduce intra- and inter-session variability of user’s brain signals. Perdikis et al [ 34 ] mostly focused on the user learning evolution by reporting the longitudinal BCI training of two tetraplegic users for the first 2016 Cybathlon event.…”

A Review on Signal Processing Approaches to Reduce Calibration Time in EEG-Based Brain–Computer Interface

“…So far, there are many TL approaches using spatial filters to generate feature representations for the raw EEG data ( Nakanishi et al, 2019 ; Zheng X. et al, 2020 ; Hehenberger et al, 2021 ). Nakanishi et al (2019) proposed a task-related component analysis (TRCA) approach in a cross-device SSVEP-based BCI speller system.…”