Amortized Q-learning with Model-based Action Proposals for Autonomous Driving on Highways

Mirchevska, Branka; Hügle, Maria; Kalweit, Gabriel; Werling, Moritz; Boedecker, Joschka

doi:10.1109/icra48506.2021.9560777

Cited by 10 publications

(13 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Recent work considers hierarchical reinforcement learning for planning in driving scenarios [1], [2], [24]. While still suffering from the pitfalls of a manually specified reward function, these approaches have the benefit that a highlevel action-selector can hand over control to safe, lowlevel, optimization-based planners.…”

Section: B Hierarchical Planningmentioning

confidence: 99%

“…While still suffering from the pitfalls of a manually specified reward function, these approaches have the benefit that a highlevel action-selector can hand over control to safe, lowlevel, optimization-based planners. Mirchevska et al use this approach to learn a high-level controller that can choose safe gaps in highway traffic for an optimization-based low-level controller to navigate to [1]. In their approach, the high-level controller only targets reachable gaps, while if a targeted gap no longer is reachable during low-level execution, control is passed back to the high-level controller.…”

Section: B Hierarchical Planningmentioning

confidence: 99%

“…We apply this hierarchical representation of occupancy measure ρ π (s, a) to reformulate the measure-matching objective in Equation (1) for policy data that is generated hierarchically:…”

Section: A Hierarchical Adversarial Imitation Learningmentioning

confidence: 99%

“…A similar option termination formulation has been used recently to facilitate handoffs between a policy learned by Q-learning to target gaps in traffic and an optimization-based controller to drive a vehicle to the chosen gap [1]. In that formulation, however, controller hand-off was deterministic and only captured the (binary) availability of a given control option.…”

Section: B Safety-aware Hierarchical Adversarial Imitation Learningmentioning

confidence: 99%

“…Recent reinforcement learning approaches add levels of redundancy to policies learned in simulation by allowing for a hierarchy of control that passes between high-level action selectors and safe low-level optimization-based driving controllers [1], [2]. Though the addition of hierarchical safety layers is intuitive and adds levels of redundancy, the success of any reinforcement learning-based approach hinges Fig.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

SHAIL: Safety-Aware Hierarchical Adversarial Imitation Learning for Autonomous Driving in Urban Environments

Arec¹,

Etienne²,

Fischer³

et al. 2022

Preprint

View full text Add to dashboard Cite

Designing a safe and human-like decision-making system for an autonomous vehicle is a challenging task. Generative imitation learning is one possible approach for automating policy-building by leveraging both real-world and simulated decisions. Previous work that applies generative imitation learning to autonomous driving policies focuses on learning a low-level controller for simple settings. However, to scale to complex settings, many autonomous driving systems combine fixed, safe, optimization-based low-level controllers with highlevel decision-making logic that selects the appropriate task and associated controller. In this paper, we attempt to bridge this gap in complexity by employing Safety-Aware Hierarchical Adversarial Imitation Learning (SHAIL), a method for learning a high-level policy that selects from a set of low-level controller instances in a way that imitates low-level driving data onpolicy. We introduce an urban roundabout simulator that controls non-ego vehicles using real data from the Interaction dataset. We then show empirically that our approach can produce better behavior than previous approaches in driver imitation which have difficulty scaling to complex environments. Our implementation is available at https://github.com/ sisl/InteractionImitation.

show abstract

Section: B Hierarchical Planningmentioning

confidence: 99%

Section: B Hierarchical Planningmentioning

confidence: 99%

“…We apply this hierarchical representation of occupancy measure ρ π (s, a) to reformulate the measure-matching objective in Equation (1) for policy data that is generated hierarchically:…”

Section: A Hierarchical Adversarial Imitation Learningmentioning

confidence: 99%

Section: B Safety-aware Hierarchical Adversarial Imitation Learningmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations