Augmenting group performance in target-face recognition via collaborative brain-computer interfaces for surveillance applications

Abstract

Abstract-This paper presents a hybrid collaborative braincomputer interface (cBCI) to improve group-based recognition of target faces in crowded scenes recorded from surveillance cameras. The cBCI uses a combination of neural features extracted from EEG and response times to estimate the decision confidence of the users. Group decisions are then obtained by weighing individual responses according to these confidence estimates. Results obtained with 10 participants indicate that the proposed cBCI improves decision errors by up to 7% over traditional group decisions based on majority. Moreover, the confidence estimates obtained by the cBCI are more accurate and robust than the confidence reported by the participants after each decision. These results show that cBCIs can be an effective means of human augmentation in realistic scenarios.

Cyborg Groups Enhance Face Recognition in Crowded Environments

“…As expected, BCI-assisted groups are significantly more accurate than equally-sized traditional groups based on standard majority for all group sizes -see "(a) vs (b)" column in Table 1. This confirms previous results obtained with this collaborative BCI both with the same set up (but using a combination of different neural features and reaction times) 40 , and in other visual-search experiments 39 . Moreover, ResNet-assisted groups were also significantly more accurate than traditional groups of only humans for all group sizes, as per "(a) vs (c)" column in Table 1.…”

“…More recently, we have used our hBCI to assist humans searching for a target face in a crowded, indoor environment 40 , which is of critical importance in many domains and is the application area considered also in this article. Once again, even in this real-world situation, groups of humans assisted by our hBCI were significantly more accurate than traditional groups based on standard majority or reported confidence 40 . While this is encouraging, with or without the assistance of an hBCI, even trained operators find this task hard and very fatiguing.…”

“…Finally, participants were asked to indicate, within 4 s, their subjective degree of confidence in that decision (0-100%) using the mouse wheel to vary the selected value in steps of 10%. Subjective confidence values were not used in this study as in preliminary analysis 40 we found that these confidence estimates were less accurate and robust than the confidence estimated by the BCI from the brain signals.…”

“…We used a state-of-the-art residual deep neural network (ResNet), our BCI estimating the decision confidence using only neural features, and human participants to create different types of cyborgs and investigate whether they can improve decision making and why. We evaluated these approaches in the context of the same face-recognition experiment as in 40 , where participants had to search an image of a crowded, indoor environment and decide whether or not it contained a target face.…”

Target Detection in Video Feeds with Selected Dyads and Groups Assisted by Collaborative Brain-Computer Interfaces

“…We used confidence estimates to weigh individual decisions and make group decisions, which were significantly better than those of individuals and equally-sized groups using standard majority. Similar results were then obtained with more realistic tasks based on static images [4], [9], [12]. We also showed that cBCI confidence estimates were better calibrated than the confidence reported by participants after each decision [9].…”