Design of Obstacle Avoidance for Autonomous Vehicle Using Deep Q-Network and CARLA Simulator

Abstract

In this paper, we propose a deep Q-network (DQN) method to develop an autonomous vehicle control system to achieve trajectory design and collision avoidance with regard to obstacles on the road in a virtual environment. The intention of this work is to simulate a case scenario and train the DQN algorithm in a virtual environment before testing it in a real scenario in order to ensure safety while reducing costs. The CARLA simulator is used to emulate the motion of the autonomous vehicle in a virtual environment, including an obstacle vehicle parked on the road while the autonomous vehicle drives on the road. The target position, real-time position, velocity, and LiDAR point cloud information are taken as inputs, while action settings such as acceleration, braking, and steering are taken as outputs. The actions are sent to the torque control in the transmission system of the vehicle. A reward function is created using continuous equations designed, especially for this case, in order to imitate human driving behaviors. The results demonstrate that the proposed method can be used to navigate to the destination without collision with the obstacle, through the use of braking and dodging methods. Furthermore, according to the trend of DQN behavior, a better result can be obtained with an increased number of training episodes. This method is a non-global path planning method successfully implemented on a virtual environment platform, which is an advantage of this method over other autonomous vehicle designs, allowing for simulation testing and application with further experiments in future work.

Deep Reinforcement Learning for Autonomous Driving in Amazon Web Services DeepRacer

“…CARLA focusses on the testing of autonomous vehicles in realistic city settings. For example, a team of researchers applied the DQN method and used the CARLA platform to emulate the motion of a self-driving vehicle within a simulation environment, which includes an obstacle vehicle [45]. Other authors also used the given simulator and collected extensive data on human drivers' responses to road obstacles to apply behaviour-cloning network architecture with the modified loss [46].…”

A Path-Planning Approach for an Unmanned Vehicle in an Off-Road Environment Based on an Improved A* Algorithm

“…In terms of path-planning algorithms, many researchers have improved many classical algorithms, including the heuristic search A* algorithm, rapidly-exploring random trees (RRT) algorithm, bionic ant colony algorithm, and machine learning Q-learning algorithm [19][20][21][22]. The principle of the RRT algorithm is to start from the starting point, adopt a tree-shaped branch structure, randomly sample the map, find the point closest to the sampling point and an accessible connection in the path tree, connect the point with the sampling point, and add the sampling point to the path tree until the area near the end point is explored [23].…”

The Development of an Autonomous Vehicle Training and Verification System for the Purpose of Teaching Experiments

“…Óscar et al [13,14] used the ROS framework through deep reinforcement learning to verify autonomous driving applications on the CARLA simulator. Terapaptommakol et al [15] proposed using a deep Q-network method in the CARLA simulator to develop an autonomous vehicle control system that achieves trajectory design and collision avoidance with obstacles on the road in a virtual environment. This approach allows for the avoidance of collisions with obstacles and enables the creation of optimized trajectories in a simulated environment.…”