Robot motion adaptation through user intervention and reinforcement learning

Jevtić, Aleksandar; Colomé, Adrià; Alenyà, Guillem; Torras, Carme

doi:10.1016/j.patrec.2017.06.017

Cited by 17 publications

(12 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Even if relational models are then mapped to execution platforms, the main difference with our work still holds: Learning is performed in a descriptive model. [56] uses RL for user-guided learning directly in the specific case of robot motion primitives.…”

Section: Related Workmentioning

confidence: 99%

Deliberative acting, planning and learning with hierarchical operational models

Patra

Mason²,

Ghallab³

et al. 2021

Artificial Intelligence

View full text Add to dashboard Cite

In AI research, synthesizing a plan of action has typically used descriptive models of the actions that abstractly specify what might happen as a result of an action, and are tailored for efficiently computing state transitions. However, executing the planned actions has needed operational models, in which rich computational control structures and closed-loop online decision-making are used to specify how to perform an action in a nondeterministic execution context, react to events and adapt to an unfolding situation. Deliberative actors, which integrate acting and planning, have typically needed to use both of these models together-which causes problems when attempting to develop the different models, verify their consistency, and smoothly interleave acting and planning.As an alternative, we define and implement an integrated acting-and-planning system in which both planning and acting use the same operational models. These rely on hierarchical task-oriented refinement methods offering rich control structures. The acting component, called Reactive Acting Engine (RAE), is inspired by the well-known PRS system. At each decision step, RAE can get advice from a planner for a near-optimal choice with respect to an utility function. The anytime planner uses a UCT-like Monte Carlo Tree Search procedure, called UPOM, whose rollouts are simulations of the actor's operational models. We also present learning strategies for use with RAE and UPOM that acquire, from online acting experiences and/or simulated planning results, a mapping from decision contexts to method instances as well as a heuristic function to guide UPOM. We demonstrate the asymptotic convergence of UPOM towards optimal methods in static domains, and show experimentally that UPOM and the learning strategies significantly improve the acting efficiency and robustness.

show abstract

Section: Related Workmentioning

confidence: 99%

Deliberative acting, planning and learning with hierarchical operational models

Patra

Mason²,

Ghallab³

et al. 2021

Artificial Intelligence

View full text Add to dashboard Cite

show abstract

“…For example, a robot may need to recognize human intent and activities based upon visual feedback (Agravante et al, 2014) or audio command (Medina et al, 2012). Another popular learning-based adaptation paradigm is reinforcement learning, which is usually designed for robot behavior adaptation (Jevtić et al, 2018; Mitsunaga et al, 2006; Ritschel and André, 2017). Recently, several methods (Kruijff- Korbayová et al, 2015; Li et al, 2015; Nikolaidis et al, 2017a,b) studied co-adaptation problems addressing how robots and humans on the same team can collaboratively adapt to each other and complete the joint task effectively.…”

Section: Related Workmentioning

confidence: 99%

Robot perceptual adaptation to environment changes for long-term human teammate following

Siva

Zhang

2020

The International Journal of Robotics Research

View full text Add to dashboard Cite

Perception is one of the several fundamental abilities required by robots, and it also poses significant challenges, especially in real-world field applications. Long-term autonomy introduces additional difficulties to robot perception, including short- and long-term changes of the robot operation environment (e.g., lighting changes). In this article, we propose an innovative human-inspired approach named robot perceptual adaptation (ROPA) that is able to calibrate perception according to the environment context, which enables perceptual adaptation in response to environmental variations. ROPA jointly performs feature learning, sensor fusion, and perception calibration under a unified regularized optimization framework. We also implement a new algorithm to solve the formulated optimization problem, which has a theoretical guarantee to converge to the optimal solution. In addition, we collect a large-scale dataset from physical robots in the field, called perceptual adaptation to environment changes (PEAC), with the aim to benchmark methods for robot adaptation to short-term and long-term, and fast and gradual lighting changes for human detection based upon different feature modalities extracted from color and depth sensors. Utilizing the PEAC dataset, we conduct extensive experiments in the application of human recognition and following in various scenarios to evaluate ROPA. Experimental results have validated that the ROPA approach obtains promising performance in terms of accuracy and efficiency, and effectively adapts robot perception to address short-term and long-term lighting changes in human detection and following applications.

show abstract

“…As an instance, the FireCommander game can be leveraged to design environments with heavy/light workload and then test how an expert's policy design efficiency and quality is affected under situational stress. Various other HRI topics can be similarly modeled to leverage FireCommander as their test-bed, be such as trust and accountability [28,29], anthropomorphism [30,31,28], human-robot co-adaptation [32], human-guided optimization [33,34,35], cognitive BCI [36,37,38,39,40] and many more [41,42].…”

Section: Stochastic and Probabilistic Environmentmentioning

confidence: 99%

FireCommander: An Interactive, Probabilistic Multi-agent Environment for Heterogeneous Robot Teams

Seraj¹,

Wu²,

Gombolay³

2020

Preprint

View full text Add to dashboard Cite

The purpose of this tutorial is to help individuals use the FireCommander game environment for research applications. The FireCommander is an interactive, probabilistic joint perceptionaction reconnaissance environment in which a composite team of agents (e.g., robots) cooperate to fight dynamic, propagating firespots (e.g., targets). In FireCommander game, a team of agents must be tasked to optimally deal with a wildfire situation in an environment with propagating fire areas and some facilities such as houses, hospitals, power stations, etc. The team of agents can accomplish their mission by first sensing (e.g., estimating fire states), communicating the sensed fire-information among each other and then taking action to put the firespots out based on the sensed information (e.g., dropping water on estimated fire locations). The FireCommander environment can be useful for research topics spanning a wide range of applications from Reinforcement Learning (RL) and Learning from Demonstration (LfD), to Coordination, Psychology, Human-Robot Interaction (HRI) and Teaming. There are four important facets of the FireCommander environment that overall, create a non-trivial game:1.

show abstract

Robot motion adaptation through user intervention and reinforcement learning

Cited by 17 publications

References 21 publications

Deliberative acting, planning and learning with hierarchical operational models

Deliberative acting, planning and learning with hierarchical operational models

Robot perceptual adaptation to environment changes for long-term human teammate following

FireCommander: An Interactive, Probabilistic Multi-agent Environment for Heterogeneous Robot Teams

Contact Info

Product

Resources

About