A Survey of End-to-End Driving: Architectures and Training Methods

Tampuu, Ardi; Semikin, Maksym; Muhammad, Naveed; Fishman, Dmytro; Matiisen, Tambet

doi:10.48550/arxiv.2003.06404

Cited by 8 publications

(12 citation statements)

References 87 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A notable example is imitation learning for lane following [2], and learning end-to-end driving from simulation [7]. We refer to [8] for a broad review. Modular methods (e.g.…”

Section: Related Workmentioning

confidence: 99%

What data do we need for training an AV motion planner?

Chen¹,

Platinsky²,

Speichert

et al. 2021

2021 IEEE International Conference on Robotics and Automation (ICRA)

View full text Add to dashboard Cite

We investigate what grade of sensor data is required for training an imitation-learning-based AV planner on human expert demonstration. Machine-learned planners [1] are very hungry for training data, which is usually collected using vehicles equipped with the same sensors used for autonomous operation [1]. This is costly and non-scalable. If cheaper sensors could be used for collection instead, data availability would go up, which is crucial in a field where data volume requirements are large and availability is small. We present experiments using up to 1000 hours worth of expert demonstration and find that training with 10x lower-quality data outperforms 1x AV-grade data in terms of planner performance (see Fig. 1). The important implication of this is that cheaper sensors can indeed be used. This serves to improve data access and democratize the field of imitation-based motion planning. Alongside this, we perform a sensitivity analysis of planner performance as a function of perception range, field-of-view, accuracy, and data volume, and reason about why lower-quality data still provide good planning results.

show abstract

“…A notable example is imitation learning for lane following [2], and learning end-to-end driving from simulation [7]. We refer to [8] for a broad review. Modular methods (e.g.…”

Section: Related Workmentioning

confidence: 99%

What data do we need for training an AV motion planner?

Chen¹,

Platinsky²,

Speichert

et al. 2021

2021 IEEE International Conference on Robotics and Automation (ICRA)

View full text Add to dashboard Cite

show abstract

“…End-to-end methods consume raw sensor data and output steering commands, for example [8] uses imitation learning for lane following, while [9] learns to drive end-to-end from simulation. We refer to [10] for a broad review. Mid-to-mid planners (e.g.…”

Section: A Machine Learning Approaches To Motion Planningmentioning

confidence: 99%

Quantity over Quality: Training an AV Motion Planner with Large Scale Commodity Vision Data

Platinsky¹,

Naseer²,

Chen³

et al. 2022

Preprint

View full text Add to dashboard Cite

With the Autonomous Vehicle (AV) industry shifting towards Autonomy 2.0 [1], the performance of self-driving systems starts to rely heavily on large quantities of expert driving demonstrations. However, collecting this demonstration data typically involves expensive HD sensor suites (LiDAR + RADAR + cameras), which quickly becomes financially infeasible at the scales required. This motivates the use of commodity vision sensors for data collection, which are an order of magnitude cheaper than the HD sensor suites, but offer lower fidelity. If it were possible to leverage these for training an AV motion planner, observing the 'long tail' of driving events would become a financially viable strategy.As our main contribution we show it is possible to train a high-performance motion planner using commodity vision data which outperforms planners trained on HD-sensor data for a fraction of the cost. We do this by comparing the autonomy system performance when training on these two different sensor configurations, and showing that we can compensate for the lower sensor fidelity by means of increased quantity: a planner trained on 100h of commodity vision data outperforms one with 25h of expensive HD data (see Fig. 1). We also share the technical challenges we had to tackle to make this work. To the best of our knowledge, we are the first to demonstrate that this is possible using real-world data.

show abstract

“…Learning action outputs is prevalent in end-to-end approaches such as [4], [30], [31]. An advantage of using action outputs for the prediction problem is that they allow to reason more closely about causal effects of actions on internal network representations.…”

Section: Related Workmentioning

confidence: 99%

“…Historically, most of the interest in human driving behavior and vehicle trajectory prediction has been shown in the context of driver assistance systems that employed classical robotics methods such as Kalman Filters (KF) [3], which perform well for short-term predictions but fail to capture intent-motivated long-term behavior. More information collected aboard vehicles, availability of large datasets, and the computing power of Graphics Processing Units (GPU) have driven the use of Deep Neural Networks (DNN) to achieve longer prediction horizons, as well as addressing full self-driving [4]. 1 Robert Bosch GmbH, Corporate Research, Advanced Autonomous Systems, 71272 Renningen, Germany.…”

Section: Introductionmentioning

confidence: 99%

Self-Supervised Action-Space Prediction for Automated Driving

Janjoš¹,

Dolgov²,

Zöllner³

2021

Preprint

View full text Add to dashboard Cite

Making informed driving decisions requires reliable prediction of other vehicles' trajectories. In this paper, we present a novel learned multi-modal trajectory prediction architecture for automated driving. It achieves kinematically feasible predictions by casting the learning problem into the space of accelerations and steering angles -by performing action-space prediction, we can leverage valuable model knowledge. Additionally, the dimensionality of the action manifold is lower than that of the state manifold, whose intrinsically correlated states are more difficult to capture in a learned manner. For the purpose of action-space prediction, we present the simple Feed-Forward Action-Space Prediction (FFW-ASP) architecture. Then, we build on this notion and introduce the novel Self-Supervised Action-Space Prediction (SSP-ASP) architecture that outputs future environment context features in addition to trajectories. A key element in the self-supervised architecture is that, based on an observed action history and past context features, future context features are predicted prior to future trajectories. The proposed methods are evaluated on real-world datasets containing urban intersections and roundabouts, and show accurate predictions, outperforming state-of-the-art for kinematically feasible predictions in several prediction metrics.

show abstract

A Survey of End-to-End Driving: Architectures and Training Methods

Cited by 8 publications

References 87 publications

What data do we need for training an AV motion planner?

What data do we need for training an AV motion planner?

Quantity over Quality: Training an AV Motion Planner with Large Scale Commodity Vision Data

Self-Supervised Action-Space Prediction for Automated Driving

Contact Info

Product

Resources

About