Anger-based BCI Using fNIRS Neurofeedback

Abstract

Functional near-infrared spectroscopy (fNIRS) holds increasing potential for Brain-Computer Interfaces (BCI) due to its portability, ease of application, robustness to movement artifacts, and relatively low cost. The use of fNIRS to support the development of affective BCI has received comparatively less attention, despite the role played by the prefrontal cortex in affective control, and the appropriateness of fNIRS to measure prefrontal activity. We present an active, fNIRS-based neurofeedback (NF) interface, which uses differential changes in oxygenation between the left and right sides of the dorsolateral prefrontal cortex to operationalize BCI input. The system is activated by users generating a state of anger, which has been previously linked to increased left prefrontal asymmetry. We have incorporated this NF interface into an experimental platform adapted from a virtual 3D narrative, in which users can express anger at a virtual character perceived as evil, causing the character to disappear progressively. Eleven subjects used the system and were able to successfully perform NF despite minimal training. Extensive analysis confirms that success was associated with the intent to express anger. This has positive implications for the design of affective BCI based on prefrontal asymmetry.

Affective Interaction with a Virtual Character Through an fNIRS Brain-Computer Interface

“…The smallest effect we could detect (with 40 s epochs sampled at 2 Hz) was d = 0.35 (small), which corresponds to 86% overlap and 60% probability of superiority. This magnitude of increase in asymmetry is comparable to our previous study, where subjects up-regulated left-asymmetry by mentally expressing anger (a negatively valenced, but approach-related affect) towards a virtual agent who was previously identified as mischievous in a narrative context (see Aranyi et al, 2015b ).…”

“…In previous work, we have demonstrated that both anger and empathy could be been successfully used in a BCI context to interact with virtual agents (Gilroy et al, 2013 ; Aranyi et al, 2015b ). Our objective here is to bridge the gap with affective expression and improve the joint analysis of objective and subjective users’ responses.…”

“…BCI input bypasses traditional expressive mechanisms of human subjects (facial expressions, gaze), while virtual agents use a range of facial expressions reflecting the user’s perceived level of engagement. We concentrate on the DL-PFC as a region of interest because of its relation to approach (Spielberg et al, 2012 ) and our previous experience with fNIRS-based NF (Aranyi et al, 2015a , b ). While our installation is not fully ecological in terms of communication, by relying on users’ mental disposition it trades naturalness for controllability and sustained interaction.…”

“…In this system, the input signal is constituted by prefrontal cortex (PFC) asymmetry measured through functional near infrared spectroscopy (fNIRS; capturing the mental disposition towards the agent), and the feedback channel is constituted by the virtual agent’s non-verbal behavior, facial expressions in particular, responding to the perceived disposition. Other supporting elements include the fact that NF facilitates BCI input where the signal is not under direct volitional control, and that PFC asymmetry has been demonstrated to be amenable to NF control (Rosenfeld et al, 1995 ; Aranyi et al, 2015b ).…”

Workload Alerts—Using Physiological Measures of Mental Workload to Provide Feedback During Tasks

“…HCI researchers are typically concerned with using fNIRS to assess workload, although some research, for example, has used fNIRS to measure anger by detecting frontal asymmetry in the PFC [2]. Measuring Mental Workload with fNIRS, however, can then be used as an additional channel of information about users during interaction with technology.…”