Predicting term-relevance from brain signals

Abstract

Term-Relevance Prediction from Brain Signals (TRPB) is proposed to automatically detect relevance of text information directly from brain signals. An experiment with forty participants was conducted to record neural activity of participants while providing relevance judgments to text stimuli for a given topic. High-precision scientific equipment was used to quantify neural activity across 32 electroencephalography (EEG) channels. A classifier based on a multi-view EEG feature representation showed improvement up to 17% in relevance prediction based on brain signals alone. Relevance was also associated with brain activity with significant changes in certain brain areas. Consequently, TRPB is based on changes identified in specific brain areas and does not require user-specific training or calibration. Hence, relevance predictions can be conducted for unseen content and unseen participants. As an application of TRPB we demonstrate a high-precision variant of the classifier that constructs sets of relevant terms for a given unknown topic of interest. Our research shows that detecting relevance from brain signals is possible and allows the acquisition of relevance judgments without a need to observe any other user interaction. This suggests that TRPB could be used in combination or as an alternative for conventional implicit feedback signals, such as dwell time or click-through activity.

Understanding Information Need

“…Another example is a study conducted by Eugster et al [17] that used EEG to show that the frequency content of the EEG signal as well as Event Related Potentials (ERPs) can be used effectively as a set of features to decode the relevance of a text. Similarly, Allegretti et al [1] reported on EEG results that indicated that within 500 ms EEG signals begin to appear that differentiate between viewing a relevant and a non relevant image.…”

The Berlin Brain-Computer Interface: Progress Beyond Communication and Control

“…Measures from eye movement patterns, pupil size, electrodermal activity (EDA), facial electromyography (fEMG), and other peripheral physiological signals can provide insights into the user's mind with respect to relevance, attention, or intent (Oliveira et al, 2009; Hardoon and Pasupa, 2010; Cole et al, 2011a,b; Gwizdka and Cole, 2011; Haji Mirza et al, 2011; Hajimirza et al, 2012; Barral et al, 2015). However, electrophysiology may provide a more direct access to the cognition of the user in comparison to eye tracking or peripheral physiology (Zander and Kothe, 2011; Eugster et al, 2014; Ušćumlić and Blankertz, 2016; Wenzel et al, 2016). …”

Video summarization using event‐related potential responses to shot boundaries in real‐time video watching

“…The authors found lower minimum potentials in brain wave responses to topic-irrelevant tags than to topic-relevant tags in the N400 time window. Eugster et al (2014) conducted an experiment wherein the neural activities of 40 subjects were recorded while they assessed relevance in response to text stimuli for a given topic. The authors observed peak differences between topic-relevant and topic-irrelevant text stimuli at the four electrodes in the interval between 450 and 800 ms.…”