Neuroadaptive technology enables implicit cursor control based on medial prefrontal cortex activity

Abstract

The effectiveness of today's human-machine interaction is limited by a communication bottleneck as operators are required to translate high-level concepts into a machine-mandated sequence of instructions. In contrast, we demonstrate effective, goal-oriented control of a computer system without any form of explicit communication from the human operator. Instead, the system generated the necessary input itself, based on real-time analysis of brain activity. Specific brain responses were evoked by violating the operators' expectations to varying degrees. The evoked brain activity demonstrated detectable differences reflecting congruency with or deviations from the operators' expectations. Real-time analysis of this activity was used to build a user model of those expectations, thus representing the optimal (expected) state as perceived by the operator. Based on this model, which was continuously updated, the computer automatically adapted itself to the expectations of its operator. Further analyses showed this evoked activity to originate from the medial prefrontal cortex and to exhibit a linear correspondence to the degree of expectation violation. These findings extend our understanding of human predictive coding and provide evidence that the information used to generate the user model is task-specific and reflects goal congruency. This paper demonstrates a form of interaction without any explicit input by the operator, enabling computer systems to become neuroadaptive, that is, to automatically adapt to specific aspects of their operator's mindset. Neuroadaptive technology significantly widens the communication bottleneck and has the potential to fundamentally change the way we interact with technology.human-computer interaction | passive brain-computer interfaces | electroencephalogram | predictive coding | neuroadaptive technology I n the European Union, 96% of enterprises rely on computers for their productivity (1). Advances in human-computer interaction (HCI), concerning the effective, efficient, and satisfying use of computer systems, may thus carry great societal benefits, e.g., in terms of productivity. However, although interaction techniques have become increasingly user-friendly-e.g., from punch cards to touch screens-they still depend on the user (operator) to translate their original thought or intention into a sequence of small, explicit commands (2). This translational step, where the human operator must ultimately obey the machine's logic, presents both a communication bottleneck and a source of potential error (3). At the same time, the computer has practically no limitation to the amount of information it can communicate, and is not as adaptable as its user. In these aspects, present-day HCI is asymmetrical (4). Comparing this to human-human interaction, Fischer (5) emphasizes the importance of a shared understanding of the situation and an understanding of the communication partner. In this sense, for a computer system to "understand" its user, it needs a model of that user-a source of informa...

Team PhyPA: Brain-Computer Interfacing for Everyday Human-Computer Interaction

“…It can be used to further narrow down the human-computer communication bottleneck by implicitly communicating information about the user to the machine, allowing it to adapt itself to the needs and aims of the user. Based on such input, neuroadaptive technology can learn more about its user over time, continuously building up and refining a user model [27]. This could ultimately lead to technology that actually understands its user, much like people understand their human communication partners in everyday communication and cooperation.…”

“…Here, implicit input, in this case information that was communicated without the participants even being aware of it, was used to control a computer cursor on a screen [27].…”

Errors and Action Monitoring: Errare Humanum Est Sed Corrigere Possibile

“…In these BCIs, the human is not required to control explicitly any device through the brain activity of interest; this activity can nevertheless be used to improve the quality of the (active or reactive) man-machine interaction by adapting the response of the machine to the mental state of the subjects (fatigue, inattention, error detection, etc. ); this occurs effortlessly for the operator who may not even be aware of it (Zander et al, 2016). Taking error potential into account in passive BCIs has proven to be extremely efficient (Ferrez and del R Millán, 2008;Chavarriaga and Millán, 2010;Chavarriaga et al, 2014;Dyson et al, 2015;Zander et al, 2016 for a review) and ''.…”

“…); this occurs effortlessly for the operator who may not even be aware of it (Zander et al, 2016). Taking error potential into account in passive BCIs has proven to be extremely efficient (Ferrez and del R Millán, 2008;Chavarriaga and Millán, 2010;Chavarriaga et al, 2014;Dyson et al, 2015;Zander et al, 2016 for a review) and ''. .…”

SEREEGA: Simulating Event-Related EEG Activity

“…Having seen almost a century of continuous research and development since its first application on humans in the 1920s (Berger, 1929), electroencephalography (EEG) is now widely used in, among others, clinical settings, neuroscience, cognitive science, psychophysiology, and brain-computer interfacing, while its use continues to expand in fields such as neuroergonomics (Parasuraman & Rizzo, 2007;Frey, Daniel, Castet, Hachet, & Lotte, 2016), neurogaming (Krol, Freytag, & Zander, 2017), neuromarketing (Vecchiato et al, 2011), neuroadaptive technology (Zander, Krol, Birbaumer, & Gramann, 2016) and mobile brain/body imaging (Gramann et al, 2011). As of December 2017, PubMed reported over 140 000 publications related to EEG, with over 4 000 published in each of the past five years.…”