Motion Planning for Mobile Robots—Focusing on Deep Reinforcement Learning: A Systematic Review

Sun, Huihui; Zhang, Weijie; Yu, Ruan; Zhang, Yujie

doi:10.1109/access.2021.3076530

Cited by 76 publications

(43 citation statements)

References 105 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…With scenarios containing moving elements, in long-range (global path planning) scenarios, the use of Evolutionary methods is adequate. The latest Artificial Intelligence methods, including the DL and RL methods, still need to be further studied to obtain solid conclusions, as also remarked by Sun et al [ 21 ]. Artificial Intelligence methods based on Fuzzy rules or neural networks can be used for fast Local Planning as an alternative to Reactive Manoeuvre methods.…”

Section: Summary and Conclusionmentioning

confidence: 96%

See 1 more Smart Citation

Path Planning for Autonomous Mobile Robots: A Review

Sánchez‐Ibáñez

Pérez-del-Pulgar

García-Cerezo

2021

Sensors

198

View full text Add to dashboard Cite

Providing mobile robots with autonomous capabilities is advantageous. It allows one to dispense with the intervention of human operators, which may prove beneficial in economic and safety terms. Autonomy requires, in most cases, the use of path planners that enable the robot to deliberate about how to move from its location at one moment to another. Looking for the most appropriate path planning algorithm according to the requirements imposed by users can be challenging, given the overwhelming number of approaches that exist in the literature. Moreover, the past review works analyzed here cover only some of these approaches, missing important ones. For this reason, our paper aims to serve as a starting point for a clear and comprehensive overview of the research to date. It introduces a global classification of path planning algorithms, with a focus on those approaches used along with autonomous ground vehicles, but is also extendable to other robots moving on surfaces, such as autonomous boats. Moreover, the models used to represent the environment, together with the robot mobility and dynamics, are also addressed from the perspective of path planning. Each of the path planning categories presented in the classification is disclosed and analyzed, and a discussion about their applicability is added at the end.

show abstract

Section: Summary and Conclusionmentioning

confidence: 96%

“…Faust et al [ 158 ] combined RL with the Probabilistic Roadmap Method (PRM), which is one of the algorithms detailed next. For more information about planning algorithms based on RL, refer to the work of Sun et al [ 21 ].…”

Section: Soft-computing-based Path Planning Algorithmsmentioning

confidence: 99%

Path Planning for Autonomous Mobile Robots: A Review

Sánchez‐Ibáñez

Pérez-del-Pulgar

García-Cerezo

2021

Sensors

198

View full text Add to dashboard Cite

show abstract

“…A jointly learnable behavior and trajectory planner for self-driving vehicles was introduced in [31]. Unlike the majority of neural planning methods that rely on paths demonstrated by experts, we apply a deep reinforcement learning approach [4], thus conserving both the time and human effort in the training phase.…”

Section: Related Workmentioning

confidence: 99%

“…Here machine learning comes to the rescue, as modern methods, like deep neural networks (DNN), make it possible to learn even complicated decision-making policies in constrained state spaces [4]. We have explored this idea in our recent paper [5], where we presented a neural network architecture and a training procedure that allow a local motion planner to learn from its own experience (Fig.…”

Section: Introductionmentioning

confidence: 99%

Speeding up deep neural network-based planning of local car maneuvers via efficient B-spline path construction

Kicki¹,

Skrzypczyński²

2022

Preprint

View full text Add to dashboard Cite

This paper demonstrates how an efficient representation of the planned path using B-splines, and a construction procedure that takes advantage of the neural network's inductive bias, speed up both the inference and training of a DNN-based motion planner. We build upon our recent work on learning local car maneuvers from past experience using a DNN architecture, introducing a novel B-spline path construction method, making it possible to generate local maneuvers in almost constant time of about 11 ms, respecting a number of constraints imposed by the environment map and the kinematics of a car-like vehicle. We evaluate thoroughly the new planner employing the recent Bench-MR framework to obtain quantitative results showing that our method outperforms stateof-the-art planners by a large margin in the considered task.

show abstract

“…The above-mentioned methods improved the disadvantages of DRL in unstructured environments, enhancing the performance of the models and improving training efficiency. The DRL-based tasks were performed by calculating cumulative rewards to obtain an optimal policy model, which would have a better performance when a large amount of high-value training samples were available [26]. However, for the fruit-picking task, there were too few valid samples at the beginning of the training due to the randomness and uncertainty of the target fruit and obstacle locations.…”

Section: Introductionmentioning

confidence: 99%

A Deep Reinforcement Learning Strategy Combining Expert Experience Guidance for a Fruit-Picking Manipulator

et al. 2022

View full text Add to dashboard Cite

When using deep reinforcement learning algorithms for path planning of a multi-DOF fruit-picking manipulator in unstructured environments, it is much too difficult for the multi-DOF manipulator to obtain high-value samples at the beginning of training, resulting in low learning and convergence efficiency. Aiming to reduce the inefficient exploration in unstructured environments, a reinforcement learning strategy combining expert experience guidance was first proposed in this paper. The ratios of expert experience to newly generated samples and the frequency of return visits to expert experience were studied by the simulation experiments. Some conclusions were that the ratio of expert experience, which declined from 0.45 to 0.35, was more effective in improving learning efficiency of the model than the constant ratio. Compared to an expert experience ratio of 0.35, the success rate increased by 1.26%, and compared to an expert experience ratio of 0.45, the success rate increased by 20.37%. The highest success rate was achieved when the frequency of return visits was 15 in 50 episodes, an improvement of 31.77%. The results showed that the proposed method can effectively improve the model performance and enhance the learning efficiency at the beginning of training in unstructured environments. This training method has implications for the training process of reinforcement learning in other domains.

show abstract

Motion Planning for Mobile Robots—Focusing on Deep Reinforcement Learning: A Systematic Review

Cited by 76 publications

References 105 publications

Path Planning for Autonomous Mobile Robots: A Review

Path Planning for Autonomous Mobile Robots: A Review

Speeding up deep neural network-based planning of local car maneuvers via efficient B-spline path construction

A Deep Reinforcement Learning Strategy Combining Expert Experience Guidance for a Fruit-Picking Manipulator

Contact Info

Product

Resources

About