Accelerating Reinforcement Learning Agent with EEG-based Implicit Human Feedback

Xu, Duo; Agarwal, Mohit; Gupta, Ekansh; Fekri, Faramarz; Sivakumar, R.

doi:10.48550/arxiv.2006.16498

Cited by 3 publications

(3 citation statements)

References 50 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…As such, our work is closer to [2], [37], [40]- [42], which used nonverbal signals to identify critical states during robot operation, detect robot errors, and adjust robot behavior. Other types of feedback signals, such as those from brain-computer interfaces, have been used in HRI [43]- [45]; however, they are impractical for navigation tasks. Simulation in HRI.…”

Section: Related Workmentioning

confidence: 99%

Recent Advances in Mobile Robot Localization in Complex Scenarios

Zhang

Liu

et al. 2023

Lecture Notes in Electrical Engineering

View full text Add to dashboard Cite

Human impressions of robot performance are often measured through surveys. As a more scalable and cost-effective alternative, we study the possibility of predicting people's impressions of robot behavior using non-verbal behavioral cues and machine learning techniques. To this end, we first contribute the SEAN TOGETHER Dataset consisting of observations of an interaction between a person and a mobile robot in a Virtual Reality simulation, together with impressions of robot performance provided by users on a 5-point scale. Second, we contribute analyses of how well humans and supervised learning techniques can predict perceived robot performance based on different combinations of observation types (e.g., facial, spatial, and map features). Our results show that facial expressions alone provide useful information about human impressions of robot performance; but in the navigation scenarios we tested, spatial features are the most critical piece of information for this inference task. Also, when evaluating results as binary classification (rather than multiclass classification), the F1-Score of human predictions and machine learning models more than doubles, showing that both are better at telling the directionality of robot performance than predicting exact performance ratings. Based on our findings, we provide guidelines for implementing these predictions models in real-world navigation scenarios.

show abstract

Section: Related Workmentioning

confidence: 99%

Recent Advances in Mobile Robot Localization in Complex Scenarios

Zhang

Liu

et al. 2023

Lecture Notes in Electrical Engineering

View full text Add to dashboard Cite

show abstract

“…X2T differs in that it learns from naturally-occurring feedback, which requires no additional effort from the user to train the agent. Other prior work trains RL agents from implicit signals, such as electroencephalography (Xu et al, 2020), peripheral pulse measurements (McDuff & Kapoor, 2019), facial expressions (Jaques et al, 2017;Cui et al, 2020), and clicks in web search (Radlinski & Joachims, 2006). X2T differs in that it trains an interface that always conditions on the user's input when selecting an action, rather than an autonomous agent that ignores user input after the training phase.…”

Section: Using the Reward Model To Select Actionsmentioning

confidence: 99%

X2T: Training an X-to-Text Typing Interface with Online Learning from User Feedback

Gao¹,

Reddy²,

Berseth³

et al. 2022

Preprint

View full text Add to dashboard Cite

We aim to help users communicate their intent to machines using flexible, adaptive interfaces that translate arbitrary user input into desired actions. In this work, we focus on assistive typing applications in which a user cannot operate a keyboard, but can instead supply other inputs, such as webcam images that capture eye gaze or neural activity measured by a brain implant. Standard methods train a model on a fixed dataset of user inputs, then deploy a static interface that does not learn from its mistakes; in part, because extracting an error signal from user behavior can be challenging. We investigate a simple idea that would enable such interfaces to improve over time, with minimal additional effort from the user: online learning from user feedback on the accuracy of the interface's actions. In the typing domain, we leverage backspaces as feedback that the interface did not perform the desired action. We propose an algorithm called x-to-text (X2T) that trains a predictive model of this feedback signal, and uses this model to fine-tune any existing, default interface for translating user input into actions that select words or characters. We evaluate X2T through a small-scale online user study with 12 participants who type sentences by gazing at their desired words, a large-scale observational study on handwriting samples from 60 users, and a pilot study with one participant using an electrocorticography-based brain-computer interface. The results show that X2T learns to outperform a non-adaptive default interface, stimulates user co-adaptation to the interface, personalizes the interface to individual users, and can leverage offline data collected from the default interface to improve its initial performance and accelerate online learning.

show abstract

“…The agent takes advantage of the implicit brain signals acquired from the human user when determining the appropriate agent action. Thus, the human user does not need to explicitly send action commands, significantly reducing the burden on the human user [7,8].…”

Section: Introductionmentioning

confidence: 99%

Error-related potential-based shared autonomy via deep recurrent reinforcement learning

Wang¹,

Chen²,

Lin³

2022

J. Neural Eng.

View full text Add to dashboard Cite

Error-related potential (ErrP)-based brain-computer interfaces (BCIs) have received a considerable amount of attention in the human-robot interaction community. In contrast to traditional BCI, which requires continuous and explicit commands from an operator, ErrP-based BCI leverages the ErrP, which is evoked when an operator observes unexpected behaviours from the robot counterpart. This paper proposes a novel shared autonomy model for ErrP-based human-robot interaction. We incorporate ErrP information provided by a user as useful observations for an agent and formulate the shared autonomy problem as a partially observable Markov decision process (POMDP). A recurrent neural network-based actor-critic model is used to address the uncertainty in the ErrP signal. We evaluate the proposed framework using a simulated human-in-the-loop robot navigation scenario with both simulated users and real users. The results show that the proposed ErrP-based shared autonomy model enables an autonomous robot to complete navigation tasks more efficiently. In a simulation with 70\% ErrP accuracy, agents completed the task 14.1% faster than in the no ErrP condition, while with real users, agents completed the navigation task 14.9 % faster.

show abstract

Accelerating Reinforcement Learning Agent with EEG-based Implicit Human Feedback

Cited by 3 publications

References 50 publications

Recent Advances in Mobile Robot Localization in Complex Scenarios

Recent Advances in Mobile Robot Localization in Complex Scenarios

X2T: Training an X-to-Text Typing Interface with Online Learning from User Feedback

Error-related potential-based shared autonomy via deep recurrent reinforcement learning

Contact Info

Product

Resources

About