Investigating static and sequential models for intervention-free selection using multimodal data of EEG and eye tracking

Salous, Mazen; Putze, Felix; Schultz, Tanja; Hild, Jutta; Beyerer, Jürgen

doi:10.1145/3279810.3279841

Cited by 3 publications

(2 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, their experiment tasks required that the participants had to find a target among non-targets. It was shown that the EEG recorded in visual search tasks and in tasks where participants look at items of different relevance to them, can help to identify the target or to measure relevance, especially due to the P300 wave, which leads to the development of a specific type of passive hybrid BCIs (Kamienkowski et al 2012, Brouwer et al 2013, Putze et al 2013, Jangraw et al 2014, Ušćumlić and Blankertz 2016, Putze et al 2016, Wenzel et al 2017, Golenia et al 2018, Salous et al 2018, Jacucci et al 2019. The P300 wave is associated with the relevance of the stimulus and the uncertainty about it.…”

Section: Introductionmentioning

confidence: 99%

A passive BCI for monitoring the intentionality of the gaze-based moving object selection

et al. 2021

View full text Add to dashboard Cite

Objective. The use of an electroencephalogram (EEG) anticipation-related component, the expectancy wave (E-wave), in brain–machine interaction was proposed more than 50 years ago. This possibility was not explored for decades, but recently it was shown that voluntary attempts to select items using eye fixations, but not spontaneous eye fixations, are accompanied by the E-wave. Thus, the use of the E-wave detection was proposed for the enhancement of gaze interaction technology, which has a strong need for a mean to decide if a gaze behavior is voluntary or not. Here, we attempted at estimating whether this approach can be used in the context of moving object selection through smooth pursuit eye movements. Approach. Eighteen participants selected, one by one, items which moved on a computer screen, by gazing at them. In separate runs, the participants performed tasks not related to voluntary selection but also provoking smooth pursuit. A low-cost consumer-grade eye tracker was used for item selection. Main results. A component resembling the E-wave was found in the averaged EEG segments time-locked to voluntary selection events of every participant. Linear discriminant analysis with shrinkage regularization classified the intentional and spontaneous smooth pursuit eye movements, using single-trial 300 ms long EEG segments, significantly above chance in eight participants. When the classifier output was averaged over ten subsequent data segments, median group ROC AUC of 0.75 was achieved. Significance. The results suggest the possible usefulness of the E-wave detection in the gaze-based selection of moving items, e.g. in video games. This technique might be more effective when trial data can be averaged, thus it could be considered for use in passive interfaces, for example, in estimating the degree of the user’s involvement during gaze-based interaction.

show abstract

Section: Introductionmentioning

confidence: 99%

A passive BCI for monitoring the intentionality of the gaze-based moving object selection

et al. 2021

View full text Add to dashboard Cite

show abstract

“…Brain computer interfaces (BCI) have been applied to motor rehabilitation in stroke patients with promising results [29]. In order to interact with different disabled patients, different modalities can be used such as manual interaction, using voice commands, gestures, eye movement, and thoughts [30]. Combining thought, gesture, eye movement, and voice as modalities for therapy allows various factors to be optimized, such as in different situations: when a disabled person is at home alone, is surrounded by people, or desires to do something him/herself [31].…”

Section: Introductionmentioning

confidence: 99%

A Blockchain-Based Non-Invasive Cyber-Physical Occupational Therapy Framework: BCI Perspective

et al. 2019

View full text Add to dashboard Cite

Although ElectroEncephaloGram (EEG) signals allow subjects suffering from neuromuscular disorders to interface their brains with the cyber-physical world, occupational therapy can be enhanced with the introduction of further modalities better assist the disabled person. In this paper, we propose an in-home occupational therapy environment, which leverages a rich set of occupational therapy-related activity recognition modalities, namely, EEG signals to understand brain activity, ElectroMyoGram (EMG) signals for muscle activity, gesture-tracking sensors for forward and inverse kinematics activities, and smart home appliance control sensors. To support a wide variety of disabled people's in-home occupational therapy, we have incorporated both selective attention and motor imagery processes for mapping a mental command with that of an occupational therapy-related command within a serious game environment. To attain higher accuracy and to avoid a higher number of false positives, a subject is first recommended to use a selective attention-based serious game in which a digital avatar of the subject acting as a model therapist will guide the therapy session. Once familiar with the generation of proper motor imagery, an advanced user can use self-paced motor imagery signals to perform occupational therapy activities within the serious game environment. The occupational therapy consists of a serious game environment in which smart home appliances are mapped with therapeutic activities through forward and inverse kinematics. The therapy data has been secured through blockchain and off-chain-based distributed repositories. The test results show the viability of using the framework in a clinical environment. INDEX TERMS Brain computer interface, blockchain, off-chain, serious games, digital virtual avatar.

show abstract

See through them

She

Jiang

et al. 2022

Proceedings of the 2nd Workshop on Games Systems

View full text Add to dashboard Cite

Investigating static and sequential models for intervention-free selection using multimodal data of EEG and eye tracking

Cited by 3 publications

References 13 publications

A passive BCI for monitoring the intentionality of the gaze-based moving object selection

A passive BCI for monitoring the intentionality of the gaze-based moving object selection

A Blockchain-Based Non-Invasive Cyber-Physical Occupational Therapy Framework: BCI Perspective

See through them

Contact Info

Product

Resources

About