End-to-End Autonomous Navigation Based on Deep Reinforcement Learning with a Survival Penalty Function

Abstract

An end-to-end approach to autonomous navigation that is based on deep reinforcement learning (DRL) with a survival penalty function is proposed in this paper. Two actor–critic (AC) frameworks, namely, deep deterministic policy gradient (DDPG) and twin-delayed DDPG (TD3), are employed to enable a nonholonomic wheeled mobile robot (WMR) to perform navigation in dynamic environments containing obstacles and for which no maps are available. A comprehensive reward based on the survival penalty function is introduced; this approach effectively solves the sparse reward problem and enables the WMR to move toward its target. Consecutive episodes are connected to increase the cumulative penalty for scenarios involving obstacles; this method prevents training failure and enables the WMR to plan a collision-free path. Simulations are conducted for four scenarios—movement in an obstacle-free space, in a parking lot, at an intersection without and with a central obstacle, and in a multiple obstacle space—to demonstrate the efficiency and operational safety of our method. For the same navigation environment, compared with the DDPG algorithm, the TD3 algorithm exhibits faster numerical convergence and higher stability in the training phase, as well as a higher task execution success rate in the evaluation phase.

Analysis of Transportation Systems for Colonies on Mars

“…The exploration of alternative transportation modes, such as hyperloops, drones, and rovers, complements the existing literature by offering a holistic view of the potential transportation landscape on Mars. The integration of RL algorithms for autonomous navigation builds upon the works of [8][9][10][11]15], showcasing the potential for advanced autonomy in enhancing the operational efficiency and adaptability of these systems in the challenging Martian environment. This contributes to the growing interest in leveraging artificial intelligence (AI) and machine learning (ML) techniques to support Martian exploration and colonization efforts.…”

“…Research by [10] on autonomous navigation for drones in complex 3D environments showcases the adaptability and efficiency of drones in overcoming the challenges posed by Martian topography and atmosphere. Drones, as detailed in [11], equipped with Deep RL capabilities, offer a promising solution for end-to-end autonomous navigation, capable of dynamically adapting to unforeseen obstacles and maximizing mission success rates. The exploration of drones for Martian applications not only complements rover missions by providing aerial perspectives and accessing otherwise unreachable areas but also opens new avenues for environmental monitoring, mapping, and supporting infrastructure development.…”

“…The simulation employs a grid-based representation of the Martian surface, where each cell can be free, occupied by an obstacle, or part of the path determined by the autonomous navigation algorithm [9,11,16]. A feature density parameter controls the distribution of obstacles across the grid, reflecting the variable complexity of the Martian terrain.…”

Path Planning of Obstacle-Crossing Robot Based on Golden Sine Grey Wolf Optimizer

“…Algorithm is the core of robot path planning [3]. The path-planning algorithm can be roughly classified as (1) path-planning algorithms based on map search, such as the A-star algorithm [4][5][6], artificial potential field algorithm [7,8], etc., (2) samplingbased path-planning algorithms, such as the RRT algorithm [9][10][11], PRM algorithm [12], etc., and (3) swarm intelligence algorithms based on global optimization, such as the ant colony optimizer [13][14][15], artificial bee colony optimizer [16,17], algorithms based on deep learning [18][19][20], etc.…”