End-to-End Urban Driving by Imitating a Reinforcement Learning Coach

Zhang, Zhejun; Liniger, Alexander; Dai, Dengxin; Yu, Fisher; Gool, Luc Van

doi:10.1109/iccv48922.2021.01494

Cited by 103 publications

(43 citation statements)

References 31 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…DA-RB+ (Prakash et al 2020) proposed an on-policy data aggregation and sampling techniques in the context of dense urban driving. Recently, Zhang et al (2021) trained an IL agent with the supervisions from an RL coach and BEV image ground-truths. In this work, we use behavior cloning tasks to help the representative feature extraction from raw observations for sub-sequent DRL-based agent rather than controlling the vehicle directly.…”

Section: Related Workmentioning

confidence: 99%

CADRE: A Cascade Deep Reinforcement Learning Framework for Vision-Based Autonomous Urban Driving

Zhao

et al. 2022

AAAI

View full text Add to dashboard Cite

Vision-based autonomous urban driving in dense traffic is quite challenging due to the complicated urban environment and the dynamics of the driving behaviors. Widely-applied methods either heavily rely on hand-crafted rules or learn from limited human experience, which makes them hard to generalize to rare but critical scenarios. In this paper, we present a novel CAscade Deep REinforcement learning framework, CADRE, to achieve model-free vision-based autonomous urban driving. In CADRE, to derive representative latent features from raw observations, we first offline train a Co-attention Perception Module (CoPM) that leverages the co-attention mechanism to learn the inter-relationships between the visual and control information from a pre-collected driving dataset. Cascaded by the frozen CoPM, we then present an efficient distributed proximal policy optimization framework to online learn the driving policy under the guidance of particularly designed reward functions. We perform a comprehensive empirical study with the CARLA NoCrash benchmark as well as specific obstacle avoidance scenarios in autonomous urban driving tasks. The experimental results well justify the effectiveness of CADRE and its superiority over the state-of-the-art by a wide margin.

show abstract

Section: Related Workmentioning

confidence: 99%

CADRE: A Cascade Deep Reinforcement Learning Framework for Vision-Based Autonomous Urban Driving

Zhao

et al. 2022

AAAI

View full text Add to dashboard Cite

show abstract

“…This approach is based on simple handcrafted rules. Building the expert with RL is also possible [111], [112] but it is more computationally demanding and less interpretable. Our expert policy consists of an A* planner followed by 2 PID controllers (for lateral and longitudinal control).…”

Section: Expertmentioning

confidence: 99%

TransFuser: Imitation with Transformer-Based Sensor Fusion for Autonomous Driving

Chitta¹,

Prakash²,

Jaeger³

et al. 2022

Preprint

View full text Add to dashboard Cite

How should we integrate representations from complementary sensors for autonomous driving? Geometry-based fusion has shown promise for perception (e.g. object detection, motion forecasting). However, in the context of end-to-end driving, we find that imitation learning based on existing sensor fusion methods underperforms in complex driving scenarios with a high density of dynamic agents. Therefore, we propose TransFuser, a mechanism to integrate image and LiDAR representations using self-attention. Our approach uses transformer modules at multiple resolutions to fuse perspective view and bird's eye view feature maps. We experimentally validate its efficacy on a challenging new benchmark with long routes and dense traffic, as well as the official leaderboard of the CARLA urban driving simulator. At the time of submission, TransFuser outperforms all prior work on the CARLA leaderboard in terms of driving score by a large margin. Compared to geometry-based fusion, TransFuser reduces the average collisions per kilometer by 48%.

show abstract

“…However, others use interpretable intermediate representations [33,34,35]. In particular, BEV semantic occupancy grid representations are widely used in modern driving approaches [36,22,23,37,26]. This representation can be inferred from images [38,39,40,41,42,26,43,44].…”

Section: Related Workmentioning

confidence: 99%

KING: Generating Safety-Critical Driving Scenarios for Robust Imitation via Kinematics Gradients

Hanselmann¹,

Renz²,

Chitta³

et al. 2022

Preprint

View full text Add to dashboard Cite

Simulators offer the possibility of safe, low-cost development of selfdriving systems. However, current driving simulators exhibit naïve behavior models for background traffic. Hand-tuned scenarios are typically added during simulation to induce safety-critical situations. An alternative approach is to adversarially perturb the background traffic trajectories. In this paper, we study this approach to safety-critical driving scenario generation using the CARLA simulator. We use a kinematic bicycle model as a proxy to the simulator's true dynamics and observe that gradients through this proxy model are sufficient for optimizing the background traffic trajectories. Based on this finding, we propose KING, which generates safety-critical driving scenarios with a 20% higher success rate than black-box optimization. By solving the scenarios generated by KING using a privileged rule-based expert algorithm, we obtain training data for an imitation learning policy. After fine-tuning on this new data, we show that the policy becomes better at avoiding collisions. Importantly, our generated data leads to reduced collisions on both held-out scenarios generated via KING as well as traditional hand-crafted scenarios, demonstrating improved robustness.

show abstract

End-to-End Urban Driving by Imitating a Reinforcement Learning Coach

Cited by 103 publications

References 31 publications

CADRE: A Cascade Deep Reinforcement Learning Framework for Vision-Based Autonomous Urban Driving

CADRE: A Cascade Deep Reinforcement Learning Framework for Vision-Based Autonomous Urban Driving

TransFuser: Imitation with Transformer-Based Sensor Fusion for Autonomous Driving

KING: Generating Safety-Critical Driving Scenarios for Robust Imitation via Kinematics Gradients

Contact Info

Product

Resources

About