Hierarchical Reinforcement Learning

Huang, Yanhua H.

doi:10.1007/978-981-15-4095-0_10

Cited by 12 publications

(20 citation statements)

References 33 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We perform experiments in CLEVR-Robot environment [Jiang et al, 2019], as shown in Figure 3. CLEVR-Robot is an environment for object interaction based on the MuJoCo physics engine [Todorov et al, 2012].…”

Section: Methodsmentioning

confidence: 99%

“…Besides, the instruction-following policy has close ties to Hierarchical RL [Barto and Mahadevan, 2003] because the instructions can be naturally viewed as a task abstraction for a low-level policy [Blukis et al, 2022]. HAL [Jiang et al, 2019] takes advantage of the compositional structure of NL and makes decisions directly at the NL level to solve long-term, complex RL tasks. These previous methods either expose the unbounded NL instructions directly to the policy or encode the NL instructions to a scenario-specific manual vector, both of which have limitations.…”

Section: Related Workmentioning

confidence: 99%

“…Specifically, we train a high-level policy outputting a TL, instructing the IFP to complete a low-level task. We consider a baseline HAL [Jiang et al, 2019] for comparison. HAL takes advantage of the compositional structure of NL and directly makes decisions on the NL level.…”

Section: Tl As An Abstraction For Hierarchical Rlmentioning

confidence: 99%

“…Concrete goal representation can be the target position to reach [Yang et al, 2022], or a target state [Nachum et al, 2018]. As for abstract goal representation, it can directly be natural language [Jiang et al, 2019] or an encoding of action primitives [Pertsch et al, 2020]. Back to the topic of this paper, language is a natural goal representation for the sub-tasks, such as open the fridge door in the above example.…”

Section: B Additional Related Workmentioning

confidence: 99%

“…Besides, we present a specific IOL implementation, including a neural network architecture that automatically discovers relationships between the objects. Through our experiments in the CLEVR-Robot environment [Jiang et al, 2019], we find that TALAR can better translate diverse NL expressions into a unique representation compared to traditional OIL methods. A policy can learn to complete human instructions efficiently and adapt to unseen NL instructions with the resulting TL.…”

Section: Introductionmentioning

confidence: 96%

See 4 more Smart Citations

Natural Language-conditioned Reinforcement Learning with Inside-out Task Language Development and Translation

Pang¹,

Yang²,

Yang³

et al. 2023

Preprint

View full text Add to dashboard Cite

Natural Language-conditioned reinforcement learning (RL) enables the agents to follow human instructions. Previous approaches generally implemented language-conditioned RL by providing human instructions in natural language (NL) and training a following policy. In this outside-in approach, the policy needs to comprehend the NL and manage the task simultaneously. However, the unbounded NL examples often bring much extra complexity for solving concrete RL tasks, which can distract policy learning from completing the task. To ease the learning burden of the policy, we investigate an inside-out scheme for natural language-conditioned RL by developing a task language (TL) that is task-related and unique. The TL is used in RL to achieve highly efficient and effective policy training. Besides, a translator is trained to translate NL into TL. We implement this scheme as TALAR (TAsk Language with predicAte Representation) that learns multiple predicates to model object relationships as the TL. Experiments indicate that TALAR not only better comprehends NL instructions but also leads to a better instruction-following policy that improves 13.4% success rate and adapts to unseen expressions of NL instruction. The TL can also be an effective task abstraction, naturally compatible with hierarchical RL.

show abstract

Section: Methodsmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Section: Tl As An Abstraction For Hierarchical Rlmentioning

confidence: 99%

Section: B Additional Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 96%

See 3 more Smart Citations

Natural Language-conditioned Reinforcement Learning with Inside-out Task Language Development and Translation

Pang¹,

Yang²,

Yang³

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

Synthesising Reward Machines for Cooperative Multi-Agent Reinforcement Learning

Varricchione,

Alechina,

Dastani

et al. 2023

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Reward machines have recently been proposed as a means of encoding team tasks in cooperative multi-agent reinforcement learning. The resulting multi-agent reward machine is then decomposed into individual reward machines, one for each member of the team, allowing agents to learn in a decentralised manner while still achieving the team task. However, current work assumes the multi-agent reward machine to be given. In this paper, we show how reward machines for team tasks can be synthesised automatically from an Alternating-Time Temporal Logic specification of the desired team behaviour and a high-level abstraction of the agents' environment. We present results suggesting that our automated approach has comparable, if not better, sample efficiency than reward machines generated by hand for multi-agent tasks.

show abstract

Introduction

Cappelen¹,

Dever²

2021

Making AI Intelligible

View full text Add to dashboard Cite

This chapter introduces the topic of the book: the philosophical foundations of AI, and in particular the powerful contemporary AI that guides our lives and receives so much attention in the media. It introduces the idea of a neural network, and poses the central puzzle of the book: when a given system’s outputs take the form of what appear to be acts of verbal communication, what is going on? When an AI system indicates, for example, that we are not creditworthy, is it saying that we are not? Can it speak? Along with posing this central question, the chapter also makes clear what the book is not about—ethics, for example—and areas in which the book might have relevance for extra-philosophical debates, such as the need for explainable AI.

show abstract

Hierarchical Reinforcement Learning

Cited by 12 publications

References 33 publications

Natural Language-conditioned Reinforcement Learning with Inside-out Task Language Development and Translation

Natural Language-conditioned Reinforcement Learning with Inside-out Task Language Development and Translation

Synthesising Reward Machines for Cooperative Multi-Agent Reinforcement Learning

Introduction

Contact Info

Product

Resources

About