Affordance-Based Human–Robot Interaction With Reinforcement Learning

Abstract

Planning precise manipulation in robotics to perform grasp and release-related operations, while interacting with humans is a challenging problem. Reinforcement learning (RL) has the potential to make robots attain this capability. In this paper, we propose an affordance-based human-robot interaction (HRI) framework, aiming to reduce the action space size that would considerably impede the exploration efficiency of the agent. The framework is based on a new algorithm called Contextual Q-learning (CQL). We first show that the proposed algorithm trains in a reduced amount of time (2.7 seconds) and reaches an 84% of success rate. This suits the robot's learning efficiency to observe the current scenario configuration and learn to solve it. Then, we empirically validate the framework for implementation in HRI real-world scenarios. During the HRI, the robot uses semantic information from the state and the optimal policy of the last training step to search for relevant changes in the environment that may trigger the generation of a new policy. INDEX TERMSQ-learning, Robotics, Affordances, Robot learning, Human-Robot Interaction [14]. Passive observation is when the robot learns from the user through video streams [15]. With teleoperation, the user

Learning to bag with a simulation‐free reinforcement learning framework for robots

“…The implementation of primitive actions relies on the perception module's analysis of RGB and depth images from the RealSense camera. Algorithm 1 contributes to determining the bag's opening area, and Equation (34) determines the evenly distributed points on the sides of the 4-sided geometry for the body of the bag. With these points, a grid can be generated inside the geometry.…”

“…First, the exploration space is reduced by implementing affordances that define what primitive actions are suitable given the current state. The use of affordances has proven to be an efficient method for reducing the exploration space [33,34]. In this work, the affordances are obtained from a manually defined set of rules denoted with Ψ.…”