Learning From the Slips of Others: Neural Correlates of Trust in Automated Agents

Abstract

With the rise of increasingly complex artificial intelligence (AI), there is a need to design new methods to monitor AI in a transparent, human-aware manner. Decades of research have demonstrated that people, who are not aware of the exact performance levels of automated algorithms, often experience a mismatch in expectations. Consequently, they will often provide either too little or too much trust in an algorithm. Detecting such a mismatch in expectations, or trust calibration, remains a fundamental challenge in research investigating the use of automation. Due to the context-dependent nature of trust, universal measures of trust have not been established. Trust is a difficult construct to investigate because even the act of reflecting on how much a person trusts a certain agent can change the perception of that agent. We hypothesized that electroencephalograms (EEGs) would be able to provide such a universal index of trust without the need of self-report. In this work, EEGs were recorded for 21 participants (mean age = 22.1; 13 females) while they observed a series of algorithms perform a modified version of a flanker task. Each algorithm’s degree of credibility and reliability were manipulated. We hypothesized that neural markers of action monitoring, such as the observational error-related negativity (oERN) and observational error positivity (oPe), are potential candidates for monitoring computer algorithm performance. Our findings demonstrate that (1) it is possible to reliably elicit both the oERN and oPe while participants monitored these computer algorithms, (2) the oPe, as opposed to the oERN, significantly distinguished between high and low reliability algorithms, and (3) the oPe significantly correlated with subjective measures of trust. This work provides the first evidence for the utility of neural correlates of error monitoring for examining trust in computer algorithms.

Grand Challenges in Neurotechnology and System Neuroergonomics

“…If users receive neurophysiological feedback over a sustained period of repeated interactions, it is possible for them to learn how to self-regulate brain activity in order to achieve a desired outcome, as would be the case with conventional neurofeedback; however, we are currently unable to assess the viability of self-regulation as an interaction mechanic beyond the mental and motor imagery protocols, which characterize active BCI (Cavazza et al, 2014). Similarly, sustained exposure to a working neurotechnology permits the user to assess an appropriate degree of trust in the system, which is likely to be highly significant for those systems associated with autonomous function (Lee and See, 2004;Hoff and Bashir, 2015) and can be assessed using psychophysiological (Hu et al, 2016) and neurophysiological (de Visser et al, 2018) measures.…”

Calibrating Trust in Automation Through Familiarity With the Autoparking Feature of a Tesla Model X

“…For that reason, there needs to be further evaluation to confirm whether this visual attentional shift changes in relation to developing trust. Future work should continue to leverage ocular and other physiological measures together with behavior to obtain a more holistic understanding of trust in human autonomy interactions, as has been done recently (Basu & Singhal, 2016;de Visser, Beatty, et al, 2018).…”

Neural Correlates of Trust During an Automated System Monitoring Task: Preliminary Results of an Effective Connectivity Study

“…Our present research seeks to replicate the findings of de Visser et al, (2018) while extending it to a more complex experimental paradigm and applying a different EEG-based analytical technique. Instead of the Eriksen flanker task, we use the automated mode of the Air-Force Multi-Attribute Task Battery (AF-MATB).…”

“…There are two problems associated with investigating and measuring trust calibration. First is the lack of a commonly accepted model of trust, even within the field of humanautomation interaction (de Visser et al, 2018). Second, most empirical work on trust has been based on psychometric measures (Dimoka, 2018), and with those come the metacognitive problem that the act of reflecting on trust may change one's own perception of it (de Visser et al, 2018).…”

“…These studies have used both EEG-and fMRI-based functional connectivity methods, and variously implicate regions of the brain related to conflict detection, mentalizing, and social cognition. Of primary importance to our study is a paper written by de Visser et al, (2018). They used EEG to show that two ERPs normally evoked when someone observes another person committing a slip (the oERN and oPe), can also be evoked when observing a computer algorithm commit a slip during the Eriksen flanker task, and that the magnitude of these ERPs was correlated with subjective ratings of trust.…”