Quick Learner Automated Vehicle Adapting its Roadmanship to Varying Traffic Cultures with Meta Reinforcement Learning

Abstract

It is essential for an automated vehicle in the field to perform discretionary lane changes with appropriate roadmanship -driving safely and efficiently without annoying or endangering other road users -under a wide range of traffic cultures and driving conditions. While deep reinforcement learning methods have excelled in recent years and been applied to automated vehicle driving policy, there are concerns about their capability to quickly adapt to unseen traffic with new environment dynamics. We formulate this challenge as a multi-Markov Decision Processes (MDPs) adaptation problem and developed Meta Reinforcement Learning (MRL) driving policies to showcase their quick learning capability. Two types of distribution variation in environments were designed and simulated to validate the fast adaptation capability of resulting MRL driving policies which significantly outperform a baseline RL.

Improved Robustness and Safety for Pre-Adaptation of Meta Reinforcement Learning with Prior Regularization

“…Optimization-based Meta-RL. This type of Meta-RLs separate learning into two loops: inner loop for policy adaptation, and outer loop for policy updates [13]. In most existing work, optimizations are done by computing gradient descents for both loops [3] [5].…”

“…2) Self-driving: Highway Merge Fig. 3: Highway-Merging Scenario Meta-RL has been applied to solve decision-making problems in autonomous driving [13] [24] [25], but none of the references we had reviewed studied the prior safety, which is an important issue for AVs.…”