EEG-Induced Autonomous Game-Teaching to a Robot Arm by Human Trainers Using Reinforcement Learning

Abstract

This paper deals with a simple indoor game, where the player has to pass a ball through a ring fixed on a variable pan-tilt platform. The motivation of the research is to learn the gaming actions of an experienced player by a robot arm for subsequent training to younger children (trainee) by the robot. The robot learns the gaming actions of the player at different game states, determined by pan-tilt orientations of the ring and its radial distance with respect to the player. The actions of the experienced player/expert are defined by six parameters: three junction-coordinates in the right arm of the player and the 3dimensional speed of the ball in a given throw. Reinforcement learning is employed here to adapt a state-action probability matrix of a probabilistic learning automation based on the reward (or penalty) scores of the player due to the success (or failure) in passing the ball through a given ring. A hybrid braincomputer interface (BCI) is used to detect the failures in the gaming action of the player by natural arousal of Error-related Potential (ErrP) signal following motor execution, indicated by motor imageries. In absence (presence) of ErrP after a motor imagination, the system considers a success (failure) in the player's trials, and thus adapts the probabilities in the learning automata according to success/failure of individual game instances. After the convergence of the state-action probability matrix, the same is used for planning, where the action corresponding to the highest probability at a given state in the automaton is selected for execution. The robot can autonomously train the game to the children using the learning automaton with converged probability scores. Experiments undertaken confirm that the success rate of the robot arm in the motor execution phase is very high (above 90%) when the ring is placed at a moderate distance of 4 feet from the robot.

Enhancing Robot Behavior with EEG, Reinforcement Learning and Beyond: A Review of Techniques in Collaborative Robotics

“…Following the line of research presented where robots learn to adapt their behavior based on error signals generated by brain waves measured by EEG sensors, there is an investigation carried out in 2021 [41] that improves the previous one by proposing an approach in which a robot arm is trained to play a game and then uses the learning to teach different children. The training process involves automatic detection of rewards and penalties based on EEG signals, probability-based action planning, and imitation of human actions for training children.…”

“…Normally in reinforcement learning, actions are planned based on a partial learning of the environment, which means that agents make decisions based on the partial knowledge they have acquired so far in the environment. In the case of the proposed research, action planning takes place after the RL algorithm has converged (convergence happens when the RL algorithm has reached a state of knowledge where it has learned enough about the environment and the actions).This approach can be very beneficial in situations where fast and accurate decisions are required, one of those could be what the article is describing: the use of RL for training a robot to play a specific game [41].…”

“…After training, the agent's behavior, particularly the robot arm's action planning, is tested. This process involves data acquisition, offline classifier training, and SAPM adaptation [41].…”

“…The second phase is online, adapting the SAPM with visual and audio stimuli for learning and correction. The test phase compares the performance of children trained by the robot to those trained by humans [41].…”

“…One of the aspects to highlight in this case is that the behavior of the robot is not directly influenced by the EEG signals, rather it will choose an action to be taken when the agent determines, either to replicate one of the movements or to throw the ball again. One of the future lines to point out could be the training of the agent in the task of throwing the ball and then, depending on the degree of error from the user, modify its behavior in a more direct way [41].…”