An Iterative Framework for EEG-based Image Search: Robust Retrieval with Weak Classifiers

Abstract

We revisit the framework for brain-coupled image search, where the Electroencephalography (EEG) channel under rapid serial visual presentation protocol is used to detect user preferences. Extending previous works on the synergy between content-based image labeling and EEG-based brain-computer interface (BCI), we propose a different perspective on iterative coupling. Previously, the iterations were used to improve the set of EEG-based image labels before propagating them to the unseen images for the final retrieval. In our approach we accumulate the evidence of the true labels for each image in the database through iterations. This is done by propagating the EEG-based labels of the presented images at each iteration to the rest of images in the database. Our results demonstrate a continuous improvement of the labeling performance across iterations despite the moderate EEG-based labeling (AUC <75%). The overall analysis is done in terms of the single-trial EEG decoding performance and the image database reorganization quality. Furthermore, we discuss the EEG-based labeling performance with respect to a search task given the same image database.

An Iterative Approach for EEG-Based Rapid Face Search: A Refined Retrieval by Brain Computer Interfaces

“…The goal of iteration in [12] is to obtain EEG labels accurate enough for the computer vision to perform further EEG label propagation. This approach quickly reaches the performance saturation within 2-3 iterations [12] and the similar result is also reported in [13]. In a recent study [13], Ušćumlić et al propose a different perspective on iterative coupling between EEG-based image labeling and computer vision.…”

“…This approach quickly reaches the performance saturation within 2-3 iterations [12] and the similar result is also reported in [13]. In a recent study [13], Ušćumlić et al propose a different perspective on iterative coupling between EEG-based image labeling and computer vision. In [13], iterations run independently, i.e., at each iteration, only the EEG-based labels obtained in the current iteration (no "the most interesting image set" is defined) are propagated to the unseen images in the database based on computer vision image similarity measure.…”

“…Studies have proposed to couple computer vision and human brain for natural image retrieval across categories [12,13], e.g. find "eagles" from "trains" and "natural scenes".…”

“…In a recent study [13], Ušćumlić et al propose a different perspective on iterative coupling between EEG-based image labeling and computer vision. In [13], iterations run independently, i.e., at each iteration, only the EEG-based labels obtained in the current iteration (no "the most interesting image set" is defined) are propagated to the unseen images in the database based on computer vision image similarity measure. The top-ranked images according to the label propagation results are composed of a new RSVP sequence for the next iteration.…”

An overview of brain computer interface

“…The features are hence distributed across space (equal to EEG channels) and frequency range [57], [56]. For ERP based paradigms the features are mostly potential values across time originating from all the EEG channels in consideration [33], [6], [64], [65], [23]. Nonetheless, both of these paradigms require feature selection methods when the data is recorded using multiple channel EEG system.…”