Options Discovery with Budgeted Reinforcement Learning

Léon, Aurélia; Denoyer, Ludovic

doi:10.48550/arxiv.1611.06824

Cited by 1 publication

(1 citation statement)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Pivotal state discovery is related to unsupervised sequence segmentation [17,47,76,13,16] and option or sub-task discovery in the context of RL [48,6,72,30,56,53,52]. Among them, both [52] and [76] employ sequential variational models to infer either task boundaries or key frames of demonstrations in an unsupervised manner, followed by applications to hierarchical RL or planning.…”

Section: Related Workmentioning

confidence: 99%

Learning World Graphs to Accelerate Hierarchical Reinforcement Learning

Shang,

Trott,

Zheng

et al. 2019

Preprint

View full text Add to dashboard Cite

In many real-world scenarios, an autonomous agent often encounters various tasks within a single complex environment. We propose to build a graph abstraction over the environment structure to accelerate the learning of these tasks. Here, nodes are important points of interest (pivotal states) and edges represent feasible traversals between them. Our approach has two stages. First, we jointly train a latent pivotal state model and a curiosity-driven goal-conditioned policy in a task-agnostic manner. Second, provided with the information from the world graph, a high-level Manager quickly finds solution to new tasks and expresses subgoals in reference to pivotal states to a low-level Worker. The Worker can then also leverage the graph to easily traverse to the pivotal states of interest, even across long distance, and explore non-locally. We perform a thorough ablation study to evaluate our approach on a suite of challenging maze tasks, demonstrating significant advantages from the proposed framework over baselines that lack world graph knowledge in terms of performance and efficiency.

show abstract