Imitation learning for agile autonomous driving

Pan, Yangdong; Cheng, Ching-An; Saigol, Kamil; Lee, Keuntaek; Yan, Xinyan; Theodorou, Evangelos A.; Boots, Byron

doi:10.1177/0278364919880273

Cited by 112 publications

(80 citation statements)

References 32 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In this context, end2end learning is defined as developing and training a complex neural network to directly map input sensory data to vehicle commands [ 10 ]. The authors of Reference [ 11 ] present an end-to-end imitation learning system for off-road autonomous driving by using only low-cost onboard sensors, having their DNN policy trained for agile driving on a predefined obstacle-free track. Since self-driving cars must manage roads with complex barriers and unclear lane borders, this strategy restricts the applicability of their system to autonomous driving.…”

Section: Related Workmentioning

confidence: 99%

OctoPath: An OcTree-Based Self-Supervised Learning Approach to Local Trajectory Planning for Mobile Robots

Trasnea

Ginerica

Zaha

et al. 2021

Sensors

View full text Add to dashboard Cite

Autonomous mobile robots are usually faced with challenging situations when driving in complex environments. Namely, they have to recognize the static and dynamic obstacles, plan the driving path and execute their motion. For addressing the issue of perception and path planning, in this paper, we introduce OctoPath, which is an encoder-decoder deep neural network, trained in a self-supervised manner to predict the local optimal trajectory for the ego-vehicle. Using the discretization provided by a 3D octree environment model, our approach reformulates trajectory prediction as a classification problem with a configurable resolution. During training, OctoPath minimizes the error between the predicted and the manually driven trajectories in a given training dataset. This allows us to avoid the pitfall of regression-based trajectory estimation, in which there is an infinite state space for the output trajectory points. Environment sensing is performed using a 40-channel mechanical LiDAR sensor, fused with an inertial measurement unit and wheels odometry for state estimation. The experiments are performed both in simulation and real-life, using our own developed GridSim simulator and RovisLab’s Autonomous Mobile Test Unit platform. We evaluate the predictions of OctoPath in different driving scenarios, both indoor and outdoor, while benchmarking our system against a baseline hybrid A-Star algorithm and a regression-based supervised learning method, as well as against a CNN learning-based optimal path planning method.

show abstract

Section: Related Workmentioning

confidence: 99%

OctoPath: An OcTree-Based Self-Supervised Learning Approach to Local Trajectory Planning for Mobile Robots

Trasnea

Ginerica

Zaha

et al. 2021

Sensors

View full text Add to dashboard Cite

show abstract

“…Recent years have witnessed a growing trend in applying deep learning techniques to autonomous driving, especially in the areas of End2End learning, as in the methods proposed by Pan et al, 3 Fan et al 7 and Bojarski et al, 2 as well as in DRL. Relevant algorithms for self-driving based on DRL can be found in the works of Kiran et al, 8 Kendal et al, 4 and Wulfmeier et al 9 Flavors of machine learning techniques have also been encountered in more traditional control approaches, such as NMPC, the uncertainty aware NMPC of Lucia and Karg 10 and the learning controllers of Ostafew et al 11 and McKinnon and Schoellig.…”

Section: Related Workmentioning

confidence: 99%

“…As shown in the deep learning for autonomous driving survey of Grigorescu et al, 1 the driving functions are traditionally implemented as perceptionplanning-action pipelines. Recently, approaches based on End2End learning from Bojarski et al 2 and Pan et al, 3 or the deep reinforcement learning (DRL) shown by Kendall et al 4 have also been proposed although mostly as research prototypes.…”

Section: Introductionmentioning

confidence: 99%

“…Synergies between data driven and classical control methods have been considered for imitation learning, where steering and acceleration control signals have been calculated in an End2End manner, as proposed by Pan et al 3 Their approach is designed for driving environments with predefined boundaries, without any obstacles present on the driving track.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

LVD-NMPC: A learning-based vision dynamics approach to nonlinear model predictive control for autonomous vehicles

Grigorescu

Ginerica²,

Zaha³

et al. 2021

International Journal of Advanced Robotic Systems

View full text Add to dashboard Cite

In this article, we introduce a learning-based vision dynamics approach to nonlinear model predictive control (NMPC) for autonomous vehicles, coined learning-based vision dynamics (LVD) NMPC. LVD-NMPC uses an a-priori process model and a learned vision dynamics model used to calculate the dynamics of the driving scene, the controlled system’s desired state trajectory, and the weighting gains of the quadratic cost function optimized by a constrained predictive controller. The vision system is defined as a deep neural network designed to estimate the dynamics of the image scene. The input is based on historic sequences of sensory observations and vehicle states, integrated by an augmented memory component. Deep Q-learning is used to train the deep network, which once trained can also be used to calculate the desired trajectory of the vehicle. We evaluate LVD-NMPC against a baseline dynamic window approach (DWA) path planning executed using standard NMPC and against the PilotNet neural network. Performance is measured in our simulation environment GridSim, on a real-world 1:8 scaled model car as well as on a real size autonomous test vehicle and the nuScenes computer vision dataset.

show abstract

“…An imitation learning based procedure for high-speed off-road driving tasks is proposed by Pan et al [ 74 ], where the policy to be mimicked is provided by a model predictive controller. The learned control policy is modeled by a deep neural network made up of two sub-networks: a CNN (fed with RGB images) and a feedforward network with a fully connected layer (fed with wheel speeds).…”

Section: End-to-end Approachesmentioning

confidence: 99%

Learning-Based Methods of Perception and Navigation for Ground Vehicles in Unstructured Environments: A Review

Guastella

Muscato

2020

Sensors

View full text Add to dashboard Cite

The problem of autonomous navigation of a ground vehicle in unstructured environments is both challenging and crucial for the deployment of this type of vehicle in real-world applications. Several well-established communities in robotics research deal with these scenarios such as search and rescue robotics, planetary exploration, and agricultural robotics. Perception plays a crucial role in this context, since it provides the necessary information to make the vehicle aware of its own status and its surrounding environment. We present a review on the recent contributions in the robotics literature adopting learning-based methods to solve the problem of environment perception and interpretation with the final aim of the autonomous context-aware navigation of ground vehicles in unstructured environments. To the best of our knowledge, this is the first work providing such a review in this context.

show abstract

Imitation learning for agile autonomous driving

Cited by 112 publications

References 32 publications

OctoPath: An OcTree-Based Self-Supervised Learning Approach to Local Trajectory Planning for Mobile Robots

OctoPath: An OcTree-Based Self-Supervised Learning Approach to Local Trajectory Planning for Mobile Robots

LVD-NMPC: A learning-based vision dynamics approach to nonlinear model predictive control for autonomous vehicles

Learning-Based Methods of Perception and Navigation for Ground Vehicles in Unstructured Environments: A Review

Contact Info

Product

Resources

About