Reinforcement Learning-Based Mapless Navigation with Fail-Safe Localisation

Lin, Feiqiang; Ji, Ze; Wei, Changyun; Niu, Hongmei

doi:10.1007/978-3-030-89177-0_10

Cited by 6 publications

(3 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In [99], a system based on DQN and reinforcement learning is designed, for which a two-dimensional simulated environment is used, unlike the present case study, which uses a three-dimensional environment. Both works use relative distances and angles for the location of the robot, and in the same way, the objective of the two works is that the robot reaches the goal.…”

Section: Discussionmentioning

confidence: 99%

Autonomous Navigation of Robots: Optimization with DQN

et al. 2023

View full text Add to dashboard Cite

In the field of artificial intelligence, control systems for mobile robots have undergone significant advancements, particularly within the realm of autonomous learning. However, previous studies have primarily focused on predefined paths, neglecting real-time obstacle avoidance and trajectory reconfiguration. This research introduces a novel algorithm that integrates reinforcement learning with the Deep Q-Network (DQN) to empower an agent with the ability to execute actions, gather information from a simulated environment in Gazebo, and maximize rewards. Through a series of carefully designed experiments, the algorithm’s parameters were meticulously configured, and its performance was rigorously validated. Unlike conventional navigation systems, our approach embraces the exploration of the environment, facilitating effective trajectory planning based on acquired knowledge. By leveraging randomized training conditions within a simulated environment, the DQN network exhibits superior capabilities in computing complex functions compared to traditional methods. This breakthrough underscores the potential of our algorithm to significantly enhance the autonomous learning capacities of mobile robots.

show abstract

Section: Discussionmentioning

confidence: 99%

Autonomous Navigation of Robots: Optimization with DQN

et al. 2023

View full text Add to dashboard Cite

show abstract

“…Gazebo has extensive sensor, robot and actuator libraries from laser range finders (Niu et al [55]), 2D/3D cameras, Kinectstyle sensors (Microsoft,[56]), contact sensors, force-torque. Many robots are provided including PR2 (Manny Ojigbo, [57]), Pioneer2 DX (Cyberbotics Ltd., [58]), iRobot Create ( iRobot Corp, [2]), Universal robot arm Liu et al [59], Kuka robot arm Niu et al 2021b [60] and TurtleBot (Open Source Robotics Foundation, Inc, [61]) (Lin et al [62]). Comparing with mobile robot and robotic arm, unmanned marine vehicle is more challenging to be simulated as it takes into account the dynamics of wind, wave, and sea current as well to help design the energy efficient control algorithm (Niu et al [63]) (Niu et al [64]) (Niu et al [65]) (Niu et al [66]) instead of just path length optimized algorithm (Niu et al [67]) (Lu et al [68]).…”

Section: Gazebo Classicmentioning

confidence: 99%

Digital Mock-Ups for Nuclear Decommissioning: A Survey on Existing Simulation Tools for Industry Applications

Caliskanelli

2023

RAEJ

View full text Add to dashboard Cite

The maturation of Virtual Reality software introduces new avenues of nuclear decommissioning research. Digital Mockups are an emerging technology which provide a virtual representation of the environment, objects or processes, supporting the whole lifecycle of product development and operations. This paper provides a survey on currently available simulation tools to design digital mock-ups required for safe remote decommissioning activities in the nuclear industry. The survey looks at eleven simulation tools; CoppeliaSim, Gazebo Classic, Gazebo (Ignition), Nvidia Omniverse Isaac Sim, WeBots, Choreonoid, AGX Dynamics, MORSE, VR4Robots, RoboDK, and Toia. Using the available documentation, the different capabilities of these software packages were assessed for their suitability to nuclear decommissioning, such as environment simulation, haptic interfaces, and general usability.

show abstract

“…Although works introduced above have obtained relatively promising results, to the authors' best knowledge, none of these learning-based algorithms have considered the effect of localisation performance on the final navigation results, as illustrated in Section I, except the preliminary work [20] by the authors. They all assume the availability of ground truth robot poses during navigation and result in policies alike shortest path strategies without considering localisation quality along navigation trajectories.…”

Section: Related Workmentioning

confidence: 99%

Localisation-Safe Reinforcement Learning for Mapless Navigation

Lin

Wei

et al. 2022

2022 IEEE International Conference on Robotics and Biomimetics (ROBIO)

Self Cite

View full text Add to dashboard Cite

Most reinforcement learning (RL)-based works for mapless point goal navigation tasks assume the availability of the robot ground-truth poses, which is unrealistic for real world applications. In this work, we remove such an assumption and deploy observation-based localisation algorithms, such as Lidar-based or visual odometry, for robot self-pose estimation. These algorithms, despite having widely achieved promising performance and being robust to various harsh environments, may fail to track robot locations under many scenarios, where observations perceived along robot trajectories are insufficient or ambiguous. Hence, using such localisation algorithms will introduce new unstudied problems for mapless navigation tasks. This work will propose a new RL-based algorithm, with which robots learn to navigate in a way that prevents localisation failures or getting trapped in local minimum regions. This ability can be learned by deploying two techniques suggested in this work: a reward metric to decide punishment on behaviours resulting in localisation failures; and a reconfigured state representation that consists of current observation and history trajectory information to transfer the problem from a partially observable Markov decision process (POMDP) to a Markov Decision Process (MDP) model to avoid local minimum.The authors thank the China Scholarship Council (CSC) for financially supporting Feiqiang Lin in his PhD programme (201906020170).

show abstract

Reinforcement Learning-Based Mapless Navigation with Fail-Safe Localisation

Cited by 6 publications

References 13 publications

Autonomous Navigation of Robots: Optimization with DQN

Autonomous Navigation of Robots: Optimization with DQN

Digital Mock-Ups for Nuclear Decommissioning: A Survey on Existing Simulation Tools for Industry Applications

Localisation-Safe Reinforcement Learning for Mapless Navigation

Contact Info

Product

Resources

About