Efficient, Swarm-Based Path Finding in Unknown Graphs Using Reinforcement Learning

Abstract

This paper addresses the problem of steering a swarm of autonomous agents out of an unknown maze to some goal located at an unknown location. This is particularly the case in situations where no direct communication between the agents is possible and all information exchange between agents has to occur indirectly through information "deposited" in the environment. To address this task, an  -greedy, collaborative reinforcement learning method using only local information exchanges is introduced in this paper to balance exploitation and exploration in the unknown maze and to optimize the ability of the swarm to exit from the maze. The learning and routing algorithm given here provides a mechanism for storing data needed to represent the collaborative utility function based on the experiences of previous agents visiting a node that results in routing decisions that improve with time. Two theorems show the theoretical soundness of the proposed learning method and illustrate the importance of the stored information in improving decision-making for routing. Simulation examples show that the introduced simple rules of learning from past experience significantly improve performance over random search and search based on Ant Colony Optimization, a metaheuristic algorithm.

Search and Rescue in a Maze-like Environment with Ant and Dijkstra Algorithms

“…Algorithm Used (Wang et al, 2017) Improved ACO (Malone et al, 2017) Artificial Potential Field Method (Atten et al, 2016) Multi Pheromones for tracking targets (Cao, 2016) Improved ACO Heuristic Function (Krentz et al, 2015) Simple ACO (Fossum et al, 2014) Repellent Pheromone for coverage (Deepak et al, 2014) Advanced PSO (Wang and Wang, 2013) Genetic Algorithm with ACO (Aurangzeb et al, 2013) Hybrid ACO w. Random-+ RL based-Search (Buniyamin et al, 2011) Point Bug Algorithm (Wang et al, 2011) Dijkstra Algorithm (Ahuja, 2010) Fuzzy Logic with Counter ACO (Gong et al, 2009) PSO in partially known environments (Sauter et al, 2005) Combination of multiple pheromones Deterministic techniques have been successfully applied to path planning (Mac et al, 2016;Yang, 2009). Some prominent techniques include searches on Visibility Graphs (VG) and Voronoi diagrams (VD) (Leena and Saju, 2014), Cell Decomposition Method (Mac et al, 2016), and gradient techniques like the Artificial Potential Field Method (Bounini et al, 2017;Khatib, 1986;Sutantyo et al, 2010), amongst others.…”

“…Therefore, the task of maze solving is essentially a path planning problem in an unknown environment. The main issues associated with maze solving include redundancy in found path Tjiharjadi and Setiawan (2016); Rivera (2012); Aurangzeb et al (2013), and premature convergence while finding local optima Bounini et al (2017) Zhangqi et al (2011). Several studies have attacked these problems using different path optimization techniques.…”