Sample Efficient Interactive End-to-End Deep Learning for Self-Driving Cars with Selective Multi-Class Safe Dataset Aggregation

Abstract

The objective of this paper is to develop a sample efficient end-to-end deep learning method for self-driving cars, where we attempt to increase the value of the information extracted from samples, through careful analysis obtained from each call to expert driver's policy. End-to-end imitation learning is a popular method for computing self-driving car policies. The standard approach relies on collecting pairs of inputs (camera images) and outputs (steering angle, etc.) from an expert policy and fitting a deep neural network to this data to learn the driving policy. Although this approach had some successful demonstrations in the past, learning a good policy might require a lot of samples from the expert driver, which might be resource-consuming. In this work, we develop a novel framework based on the Safe Dataset Aggregation (safe DAgger) approach, where the current learned policy is automatically segmented into different trajectory classes, and the algorithm identifies trajectory segments/classes with the weak performance at each step. Once the trajectory segments with weak performance identified, the sampling algorithm focuses on calling the expert policy only on these segments, which improves the convergence rate. The presented simulation results show that the proposed approach can yield significantly better performance compared to the standard Safe DAgger algorithm while using the same amount of samples from the expert.

Automated Lane Change Decision Making using Deep Reinforcement Learning in Dynamic and Uncertain Highway Environment

“…(6). a n > b saf e (6) whereã n refers to the new acceleration of the follower after making a lane change and b saf e is the maximum safe deceleration. Secondly, if the first safety criterion is met, MOBIL checks for the second criterion, which is a collection of acceleration gains of surrounding vehicles as in Eq.…”

“…Since this work is using the learning methods, the surveyed literature will be narrowed to the ML-based approaches. Various learning approaches from end-to-end imitation learning [4], [5], [6] to Deep reinforcement learning (deep RL) [7] have been applied to autonomous vehicles. Deep RL is efficient in learning arbitrary policies defining specific goals.…”

DeFIX: Detecting and Fixing Failure Scenarios with Reinforcement Learning in Imitation Learning Based Autonomous Driving

“…In the domain of autonomous driving, particularly Learning by Cheating (LBC) [4] applied an IL method in two steps, training privileged agents using ground truth information from the simulator and training a policy agent by cloning the behavior of a privileged agent. For effective utilization of expert demonstrator, as presented in [8], the usage of Data Aggregation (DAgger) [6] technique in end-to-end autonomous driving tasks yields better performance boosts with fewer data required.…”

Self-Improving Safety Performance of Reinforcement Learning Based Driving with Black-Box Verification Algorithms

“…In a recent study [3], researchers reduced unsafe scenarios of a black-box system by guiding exploration samples on predefined trajectory classes. Without verification on a rareevent simulation [4] and generalized importance sampling on a continuous action and observation space, the safety of the AD system could not be guaranteed.…”