Simulation for path planning of autonomous underwater vehicle using Flower Pollination Algorithm, Genetic Algorithm and Q-Learning

Abstract

the motivation behind this paper is to address the necessity for exploration in near bottom ocean environment employing Autonomous Underwater Vehicles. This paper presents a simulation for an optimized path planning for an autonomous underwater vehicle in benthic ocean zones. The statistical data pertaining to the near-bottom ocean currents has been sourced from the Bedford Institute of Oceanography, Canada. A cost function is developed which incorporates the interaction of the underwater vehicle with the ocean currents. This cost function takes the source and destination coordinates as the inputs and outputs the time taken by the vehicle to travel between them. This paper aims to minimize this cost function to obtain a path having the least travel time for the vehicle. Various biologically inspired algorithms such as Flower Pollination Algorithm and Genetic Algorithm have been used to optimize this cost function. The optimization of the cost function has also been performed using Q-Learning technique and the results have been compared with the biologically inspired algorithms. The results depict that Q-Learning Algorithm is better in computational complexity and ease of simulating the environment. Thus, an efficient Path planning technique, which has been tested for the cost function of an autonomous underwater vehicle is proposed through this paper.

Semantic-Aware Path Planning with Hexagonal Grids and Vehicle Dynamic Constraints

“…For route planning in a dynamic environment using genetic algorithms, see [5]. The combination of genetic algorithms and Q-learning is described in [6]. Another example of using Q-learning for a dynamic environment is in [7,8].…”

“…This approach is not sufficient. Equations ( 5) and (6) show that whether a given terrain is passable depends not only on the slope of the elevation but also on the type of surface, the type of wheels, and the engine power. In our approach, each map grid contains two values: the terrain height and the semantic label.…”

Path Planning for Wheeled Mobile Robot in Partially Known Uneven Terrain

“…Then, the robot’s action with the maximum Q value at each current state can be chosen until reaching the goal position. Based on the Q -learning algorithm designed in [ 32 ], when the robot performs an action in state , the corresponding action value function can be updated as where is the attenuation rate representing the attenuation of future rewards, and is the learning rate. The attenuation rate affects the ratio that the robot replaces the original Q value with a new value.…”

“…Then, the robot's action with the maximum Q value at each current state can be chosen until reaching the goal position. Based on the Q-learning algorithm designed in [32], when the robot performs an action a t in state s t , the corresponding action value function Q t (s t , a t ) can be updated as…”

Learning‐Based Path Planning Algorithm in Ocean Currents for Multi‐Glider

“…Reinforcement learning is a kind of unsupervised machine learning, and its learning can be regarded as a trial evaluation process [28]. Considering the impact of ocean current on the energy consumption of underwater gliders, the design considers the cost function of ocean current to optimize the path planning in the 3D environment [29]. Compared with RL, which focuses on solving learning problems, deep learning networks can extract abstract features from large-scale data to cope with increasingly complex task environments.…”