Deep Reinforcement Learning for Robot Collision Avoidance With Self-State-Attention and Sensor Fusion

Han, Yiheng; Zhan, Irvin Haozhe; Zhao, Wang; Pan, Jia; Zhang, Ziyang; Wang, Yaoyuan; Liu, Yong-Jin

doi:10.1109/lra.2022.3178791

Cited by 34 publications

(6 citation statements)

References 37 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…While the laser range data provide information about the distance of obstacles, they do not provide spatial information near the robot, which is implicit in the image data of an red, green, blue (RGB) or red, green, blue, depth map (RGBD) camera. Therefore, overcoming the sim‐to‐real gap in image data [ 33 ] or fusing laser range data with image data [ 34 ] is a promising research topic in DRL‐based path planning methods. In addition, the training effect improved using LSTM is considerable; thus, the combination of the BOAE mechanism and recurrent neural networks is worth exploring.…”

Section: Discussionmentioning

confidence: 99%

Bidirectional Obstacle Avoidance Enhancement‐Deep Deterministic Policy Gradient: A Novel Algorithm for Mobile‐Robot Path Planning in Unknown Dynamic Environments

Xue,

Zhang,

et al. 2024

Advanced Intelligent Systems

View full text Add to dashboard Cite

Real‐time path planning in unknown dynamic environments is a significant challenge for mobile robots. Many researchers have attempted to solve this problem by introducing deep reinforcement learning, which trains agents through interaction with their environments. A method called BOAE‐DDPG, which combines the novel bidirectional obstacle avoidance enhancement (BOAE) mechanism with the deep deterministic policy gradient (DDPG) algorithm, is proposed to enhance the learning ability of obstacle avoidance. Inspired by the analysis of the reaction advantage in dynamic psychology, the BOAE mechanism focuses on obstacle‐avoidance reactions from the state and action. The cross‐attention mechanism is incorporated to enhance the attention to valuable obstacle‐avoidance information. Meanwhile, the obstacle‐avoidance behavioral advantage is separately estimated using the modified dueling network. Based on the learning goals of the mobile robot, new assistive reward factors are incorporated into the reward function to promote learning and convergence. The proposed method is validated through several experiments conducted using the simulation platform Gazebo. The results show that the proposed method is suitable for path planning tasks in unknown environments and has an excellent obstacle‐avoidance learning capability.

show abstract

Section: Discussionmentioning

confidence: 99%

Bidirectional Obstacle Avoidance Enhancement‐Deep Deterministic Policy Gradient: A Novel Algorithm for Mobile‐Robot Path Planning in Unknown Dynamic Environments

Xue,

Zhang,

et al. 2024

Advanced Intelligent Systems

View full text Add to dashboard Cite

show abstract

“…In research [113], a straightforward yet efficient deep reinforcement learning (DRL) approach featuring our self-state-attention unit is introduced. A solution that enables the navigation of a tall mobile robot, measuring one meter in height, using low-cost devices such as a 2D LiDAR sensor and a monocular camera is proposed.…”

Section: Senor Fusionmentioning

confidence: 99%

A Review of Path Planning Methods for Marine Autonomous Surface Vehicles

Wu,

Wang,

Liu

2024

JMSE

View full text Add to dashboard Cite

A marine autonomous surface vehicle (ASV) is a kind of autonomous marine robot with intelligent and flexible use advantages. They are mainly divided into two categories: unmanned vessels and unmanned sailboats. Marine ASVs are essential in marine science, industry, environmental protection, and national defense. One of the primary challenges faced by marine ASVs is autonomously planning paths in an intricate marine environment. Numerous research findings have surfaced in recent years, including the combination with popular machine learning. However, a systematic literature review is still lacking, primarily a comprehensive comparison of two types of ASV path planning methods. This review first introduces the problem and evaluation indicators of path planning for ASVs. Then, aiming at unmanned vessels and sailboats, respectively, it sorts out various path planning algorithms proposed in the existing literature, including the advantages and limitations of both kinds of ASVs, and discusses them in combination with evaluation indicators. Also, this paper explores how marine environmental factors affect path planning and its corresponding treatment methods. Finally, this review summarizes the challenges of unmanned ship path planning, proposes potential technical solutions and future development directions, and aims to provide references for further development in this field.

show abstract

“…Tai et al [33] developed a navigation learning model in a simulated environment using a 10-dimensional laser beam as one input to the model. Han et al [34] used the fusion of RGB images from a camera and 2D LiDAR sensor data as input to a DRL network of self-state attention to investigate the effect of using 2D LiDAR on a tall robot. In their work, the training environment is captured and processed before passing it to the training network.…”

Section: Related Workmentioning

confidence: 99%

The Impact of LiDAR Configuration on Goal-Based Navigation within a Deep Reinforcement Learning Framework

Olayemi,

Van,

McLoone

et al. 2023

Sensors

View full text Add to dashboard Cite

Over the years, deep reinforcement learning (DRL) has shown great potential in mapless autonomous robot navigation and path planning. These DRL methods rely on robots equipped with different light detection and range (LiDAR) sensors with a wide field of view (FOV) configuration to perceive their environment. These types of LiDAR sensors are expensive and are not suitable for small-scale applications. In this paper, we address the performance effect of the LiDAR sensor configuration in DRL models. Our focus is on avoiding static obstacles ahead. We propose a novel approach that determines an initial FOV by calculating an angle of view using the sensor’s width and the minimum safe distance required between the robot and the obstacle. The beams returned within the FOV, the robot’s velocities, the robot’s orientation to the goal point, and the distance to the goal point are used as the input state to generate new velocity values as the output action of the DRL. The cost function of collision avoidance and path planning is defined as the reward of the DRL model. To verify the performance of the proposed method, we adjusted the proposed FOV by ±10° giving a narrower and wider FOV. These new FOVs are trained to obtain collision avoidance and path planning DRL models to validate the proposed method. Our experimental setup shows that the LiDAR configuration with the computed angle of view as its FOV performs best with a success rate of 98% and a lower time complexity of 0.25 m/s. Additionally, using a Husky Robot, we demonstrate the model’s good performance and applicability in the real world.

show abstract

Deep Reinforcement Learning for Robot Collision Avoidance With Self-State-Attention and Sensor Fusion

Cited by 34 publications

References 37 publications

Bidirectional Obstacle Avoidance Enhancement‐Deep Deterministic Policy Gradient: A Novel Algorithm for Mobile‐Robot Path Planning in Unknown Dynamic Environments

Bidirectional Obstacle Avoidance Enhancement‐Deep Deterministic Policy Gradient: A Novel Algorithm for Mobile‐Robot Path Planning in Unknown Dynamic Environments

A Review of Path Planning Methods for Marine Autonomous Surface Vehicles

The Impact of LiDAR Configuration on Goal-Based Navigation within a Deep Reinforcement Learning Framework

Contact Info

Product

Resources

About