Learning Options from Demonstration using Skill Segmentation

Cockcroft, Matthew; Mawjee, Shahil; James, Steven; Ranchod, Pravesh

doi:10.1109/saupec/robmech/prasa48453.2020.9040988

Cited by 2 publications

(2 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Agents can utilize observations of exemplar behavior in order to learn aspects of the desired model. For instance, agents can learn actions or skills from observed imagery via the inclusion of a learned cost function [24,124] or by clustering observations into skills [22]. Other recent results have focused on analytical/theoretical aspects of the problem relating to reward function search or aspects of the behavior policy providing the demonstrations [10,50,79,100].…”

Section: Applications and Recent Resultsmentioning

confidence: 99%

Modeling Human Behavior Part I -- Learning and Belief Approaches

Fuchs¹,

Passarella²,

Conti³

2022

Preprint

View full text Add to dashboard Cite

There is a clear desire to model and comprehend human behavior. Trends in research covering this topic show a clear assumption that many view human reasoning as the presupposed standard in artificial reasoning. As such, topics such as game theory, theory of mind, machine learning, etc. all integrate concepts which are assumed components of human reasoning. These serve as techniques to attempt to both replicate and understand the behaviors of humans. In addition, next generation autonomous and adaptive systems will largely include AI agents and humans working together as teams. To make this possible, autonomous agents will require the ability to embed practical models of human behavior, which allow them not only to replicate human models as a technique to "learn", but to to understand the actions of users and anticipate their behavior, so as to truly operate in symbiosis with them. The main objective of this paper it to provide a succinct yet systematic review of the most important approaches in two areas dealing with quantitative models of human behaviors. Specifically, we focus on (i) techniques which learn a model or policy of behavior through exploration and feedback, such as Reinforcement Learning, and (ii) directly model mechanisms of human reasoning, such as beliefs and bias, without going necessarily learning via trial-and-error.CCS Concepts: • Human-centered computing; • General and reference → Surveys and overviews;

show abstract

Section: Applications and Recent Resultsmentioning

confidence: 99%

Modeling Human Behavior Part I -- Learning and Belief Approaches

Fuchs¹,

Passarella²,

Conti³

2022

Preprint

View full text Add to dashboard Cite

show abstract

“…For a skill to be obtained autonomously, the components of the triplet ⟨I, , ⟩ are needed. Therefore, there are many HRL studies that focus on detecting subgoals and/or defining initiation sets (Xu et al 2018;Simsek and Barreto 2008;Davoodabadi and Beigy 2011a, b;Shoeleh and Asadpour 2017;Kazemitabar et al 2018;Farahani and Mozayani 2019;Machado et al 2017;McGovern and Barto 2001;Stolle and Precup 2002;Şimşek and Barto 2004a;Ghafoorian et al 2013;Daniel et al 2016;Cockcroft et al 2020). Graph theoretic approaches are widely used for subgoal detection (Xu et al 2018;Simsek and Barreto 2008;Davoodabadi and Beigy 2011a, b;Shoeleh and Asadpour 2017;Kazemitabar et al 2018;Farahani and Mozayani 2019;Machado et al 2017).…”

Section: Related Workmentioning

confidence: 99%

Autonomous acquisition of arbitrarily complex skills using locality based graph theoretic features: a syntactic approach to hierarchical reinforcement learning

et al. 2023

View full text Add to dashboard Cite

With the growing state/action space, learning a satisfactory policy for regular Reinforcement Learning (RL) algorithms such as flat Q-learning becomes quickly infeasible. One possible solution to handle such cases is to employ hierarchical RL (HRL). In this work, we present two methods to autonomously construct (1) skills (ASKA) and (2) arbitrarily elaborate superskills or complexes through defining an arbitrary number of hierarchies in HRL (ASKAC) over a graph-based iteratively-growing environment model. We employ dynamic community detection (DCD) in detecting subgoals since DCD considers local changes only over the partially growing graphs and lowers the time complexity of the subgoal detection where groups of environment states (i.e., subenvironments) are modeled by communities from the graph theory. DCD's drawback is oversegmentation where it mispartitions a subenvironment further into smaller components. To maintain the robustness of ASKAC against DCD's possible oversegmentation we introduce the concept of skill coupling. Skill coupling does not only robustly solve the oversegmentation issue, but it also improves HRL by building up more elaborate complexes (i.e., skill compositions) obtained at an arbitrary number of hierarchies and reduces the number of decisions leading to the goal employing these complexes. In addition to the experiments that investigate the effect of parameters, proposed methods are experimentally evaluated in grid world and taxi driver benchmark environments.

show abstract

Learning Options from Demonstration using Skill Segmentation

Cited by 2 publications

References 14 publications

Modeling Human Behavior Part I -- Learning and Belief Approaches

Modeling Human Behavior Part I -- Learning and Belief Approaches

Autonomous acquisition of arbitrarily complex skills using locality based graph theoretic features: a syntactic approach to hierarchical reinforcement learning

Contact Info

Product

Resources

About