Deep Learning Based Motion Planning For Autonomous Vehicle Using Spatiotemporal LSTM Network

Abstract

Motion Planning, as a fundamental technology of automatic navigation for autonomous vehicle, is still an open challenging issue in real-life traffic situation and is mostly applied by the model-based approaches. However, due to the complexity of the traffic situations and the uncertainty of the edge cases, it is hard to devise a general motion planning system for autonomous vehicle. In this paper, we proposed a motion planning model based on deep learning (named as spatiotemporal LSTM network), which is able to generate a real-time reflection based on spatiotemporal information extraction. To be specific, the model based on spatiotemporal LSTM network has three main structure. Firstly, the Convolutional Long-short Term Memory (Conv-LSTM) is used to extract hidden features through sequential image data. Then, the 3D Convolutional Neural Network(3D-CNN) is applied to extract the spatiotemporal information from the multi-frame feature information. Finally, the fully connected neural networks are used to construct a control model for autonomous vehicle steering angle. The experiments demonstrated that the proposed method can generate a robust and accurate visual motion planning results for autonomous vehicle.

Hierarchical Reinforcement Learning for Autonomous Decision Making and Motion Planning of Intelligent Vehicles

“…s t as in equation (16) or (23). Then we use the trained policy w π to make real-time decisions by equations (14) and (15). In the aspect of motion planning, as the decisions have included the longitudinal speed information, we only concern about the lateral trajectory planning by equation (26), i.e.…”

“…Kernel-based API in [8] and ELM-API in [9] dealt with simple overtaking tasks and lane-change issues without considering the trajectory planning when there exists the obstacles. In [10]- [14], deep learning approaches were used for training the collected raw sensor samples in specific scenarios which can map the images to actions. The advantages of using deep neural networks include automatic feature extraction and feature representation.…”

“…end if 13: end for This means the algorithm complexity grows linearly with sample numbers T , and the computation time is also related to the number of kernel centers d. Remark 3: When the trained policy is deployed for realtime decision-making problems, the required flops for computing the optimal policy (i.e. equations (14) and (15)) are n a (2d − 1), where n a = 3 is the number of optional actions. The flops count can be reduced approximately as d. The number of kernel centers d in scenario-A is with a magnitude of 10 2 , while is about 10 3 in scenario-B.…”

Object Detection, Distributed Cloud Computing and Parallelization Techniques for Autonomous Driving Systems

“…Currently, there are many approaches to road prediction using LSTM architectures for this type of forecasts. Bai et al [67], suggested spatiotemporal LSTM architecture for motion planning in AVs. It uses convolutional layers for feature extraction in an image, and it runs through an LSTM to find sequences in those images, something similar to a CRNN architecture.…”

Deep, Consistent Behavioral Decision Making with Planning Features for Autonomous Vehicles

“…Another class of methods focuses on learning driving policy from raw sensor inputs, such as those from cameras, LIDAR and screen shots of a simulator [3][4][5][20][21][22]. Sallab [20] et al proposed a new framework of DRL for autonomous vehicles, which combines LSTM and attention networks to capture the spatial features.…”