On Localisation Errors and Cooperative Search for a Swarm of AUVs

El-Mihoub, Tarek A.; Nolle, Lars; Tholen, Christoph

doi:10.1109/ieeeconf38699.2020.9389227

Cited by 2 publications

(1 citation statement)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Researchers in this group design various agent-based interaction mechanisms as the behavior guideline for each robot to follow. While each robot is executing its own dynamics, the robot swarm, as a team, is exhibiting a certain group-level behavior for efficient target search [7], [13], [15], [36], [37], [38]. For example, Tang et al [36] revise the grey wolf optimization method to dynamically decide the next-goal point for each individual robot; with the next goal information, each robot also takes its observed obstacle information and momentum into consideration and reaches the finally merged dynamics.…”

Section: B Mures Methodologiesmentioning

confidence: 99%

Cross-Entropy Regularized Policy Gradient for Multirobot Nonadversarial Moving Target Search

et al. 2023

View full text Add to dashboard Cite

This article investigates the multirobot efficient search (MuRES) for a nonadversarial moving target problem from the multiagent reinforcement learning (MARL) perspective. MARL is deemed as a promising research field for cooperative multiagent applications. However, one of the main bottlenecks of applying MARL to the MuRES problem is the nonstationarity introduced by multiple learning agents. With learning agents simultaneously updating their policies, the environment cannot be modeled as a stationary Markov decision process, which results in the inapplicability of fundamental reinforcement learning techniques such as deep Q-network and policy gradient (PG). In view of that, we adopt the centralized training and decentralized execution scheme and thereby propose a cross-entropy regularized policy gradient (CE-PG) method to train the learning agents/robots. We let the robots commit to a predetermined policy during execution, collect the trajectories, and then perform centralized training for the corresponding policy improvement. In this way, the nonstationarity problem is overcome, in that the robots do not update their policies during execution. During the centralized training stage, we improve the canonical PG method to consider the interactions among robots by adding a cross-entropy regularization term, which essentially functions to "disperse" the robots in the environment. Extensive simulation results and comparisons with state of the art show CE-PG's superior performance, and we also validate the algorithm with a real multirobot system in an indoor moving target search scenario.

show abstract

Section: B Mures Methodologiesmentioning

confidence: 99%

Cross-Entropy Regularized Policy Gradient for Multirobot Nonadversarial Moving Target Search

et al. 2023

View full text Add to dashboard Cite

show abstract

Artificial Intelligence Search Strategies for Autonomous Underwater Vehicles Applied for Submarine Groundwater Discharge Site Investigation

Tholen

El-Mihoub

Nolle

2021

JMSE

View full text Add to dashboard Cite

In this study, a set of different search strategies for locating submarine groundwater discharge (SGD) are investigated. This set includes pre-defined path planning (PPP), adapted random walk (RW), particle swarm optimisation (PSO), inertia Levy-flight (ILF), self-organising-migration-algorithm (SOMA), and bumblebee search algorithm (BB). The influences of self-localisation and communication errors and limited travel distance of the autonomous underwater vehicles (AUVs) on the performance of the proposed algorithms are investigated. This study shows that the proposed search strategies could not outperform the classic search heuristic based on full coverage path planning if all AUVs followed the same search strategy. In this study, the influence of self-localisation and communication errors was investigated. The simulations showed that, based on the median error of the search runs, the performance of SOMA was in the same order of magnitude regardless the strength of the localisation error. Furthermore, it was shown that the performance of BB was highly affected by increasing localisation errors. From the simulations, it was revealed that all the algorithms, except for PSO and SOMA, were unaffected by disturbed communications. Here, the best performance was shown by PPP, followed by BB, SOMA, ILF, PSO, and RW. Furthermore, the influence of the limited travel distances of the AUVs on the search performance was evaluated. It was shown that all the algorithms, except for PSO, were affected by the shorter maximum travel distances of the AUVs. The performance of PPP increased with increasing maximum travel distances. However, for maximum travel distances > 1800 m the median error appeared constant. The effect of shorter travel distances on SOMA was smaller than on PPP. For maximum travel distances < 1200 m, SOMA outperformed all other strategies. In addition, it can be observed that only BB showed better performances for shorter travel distances than for longer ones. On the other hand, with different search strategies for each AUV, the search performance of the whole swarm can be improved by incorporating population-based search strategies such as PSO and SOMA within the PPP scheme. The best performance was achieved for the combination of two AUVs following PPP, while the third AUV utilised PSO. The best fitness of this combination was 15.9. This fitness was 26.4% better than the performance of PPP, which was 20.4 on average. In addition, a novel mechanism for dynamically selecting a search strategy for an AUV is proposed. This mechanism is based on fuzzy logic. This dynamic approach is able to perform at least as well as PPP and SOMA for different travel distances of AUVs. However, due to the better adaptation to the current situation, the overall performance, calculated based on the fitness achieved for different maximum travel distances, the proposed dynamic search strategy selection performed 32.8% better than PPP and 34.0% better than SOMA.

show abstract

On Localisation Errors and Cooperative Search for a Swarm of AUVs

Cited by 2 publications

References 17 publications

Cross-Entropy Regularized Policy Gradient for Multirobot Nonadversarial Moving Target Search

Cross-Entropy Regularized Policy Gradient for Multirobot Nonadversarial Moving Target Search

Artificial Intelligence Search Strategies for Autonomous Underwater Vehicles Applied for Submarine Groundwater Discharge Site Investigation

Contact Info

Product

Resources

About