Local Roots: A Tree-Based Subgoal Discovery Method to Accelerate Reinforcement Learning

Demіr, Alper; Çilden, Erkin; Polat, Faruk

doi:10.1007/978-3-319-46227-1_23

Cited by 4 publications

(1 citation statement)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The relative novelty [11] assumes that subgoals can lead the agent to access a new region from highly visited regions. The Local Roots algorithm [12] considers that subgoals should be junctions of shortcut paths from each state to the goal state. This approach online constructs a sequence tree from collected successful trajectories and takes the one with the local maximum root factor measure as subgoal.…”

Section: Literature Reviewmentioning

confidence: 99%

Constructing Temporally Extended Actions through Incremental Community Detection

Mei

2018

Computational Intelligence and Neuroscience

View full text Add to dashboard Cite

Hierarchical reinforcement learning works on temporally extended actions or skills to facilitate learning. How to automatically form such abstraction is challenging, and many efforts tackle this issue in the options framework. While various approaches exist to construct options from different perspectives, few of them concentrate on options' adaptability during learning. This paper presents an algorithm to create options and enhance their quality online. Both aspects operate on detected communities of the learning environment's state transition graph. We first construct options from initial samples as the basis of online learning. Then a rule-based community revision algorithm is proposed to update graph partitions, based on which existing options can be continuously tuned. Experimental results in two problems indicate that options from initial samples may perform poorly in more complex environments, and our presented strategy can effectively improve options and get better results compared with flat reinforcement learning.

show abstract

Section: Literature Reviewmentioning

confidence: 99%