Communication-Efficient and Collision-Free Motion Planning of Underwater Vehicles via Integral Reinforcement Learning

Abstract

Motion planning of underwater vehicles is regarded as a promising technique to make up the flexibility deficiency of underwater sensor networks (USNs). Nonetheless, the unique characteristics of underwater channel and environment make it challenging to achieve the above mission. This article is concerned with a communication-efficient and collision-free motion planning issue for underwater vehicles in fading channel and obstacle environment. We first develop a model-based integral reinforcement learning (IRL) estimator to predict the stochastic signal-to-noise ratio (SNR). With the estimated SNR, an integrated optimization problem for the codesign of communication efficiency and motion planning is constructed, in which the underwater vehicle dynamics, communication capacity, collision avoidance, and position control are all considered. In order to tackle this problem, a model-free IRL algorithm is designed to drive underwater vehicles to the desired position points while maximizing the communication capacity and avoiding the collision. It is worth mentioning that, the proposed motion planning solution in this article considers a realistic underwater communication channel, as well as a realistic dynamic model for underwater vehicles. Finally, simulation and experimental results are demonstrated to verify the effectiveness of the proposed approach.

Communication-Aware Mobile Relaying via an AUV for Minimal Wait Time: A Broad Learning-Based Solution

“…Related works: Recently, the mobile relaying of an AUV has been widely used in marine applications [1], [2]. Since the distribution of underwater channels is uneven, some scholars are committed to use the spatial distribution of channel quality to guide an AUV to relay data in different positions, e.g., [3], [4]. However, the above algorithms ignore the impact of obstacles on channel distribution.…”

Communication-Aware Motion Planning of AUV in Obstacle-Dense Environment: A Binocular Vision-Based Deep Learning Method

“…Note that the random parameters σ S H and µ M P in (2) are unknown to AUV. In our previous work [31], [32], an integral reinforcement learning-based estimator was developed to capture the unknown shadowing and multipath parameters. For that reason, we employ our previous channel estimation approach to predict the SNR.…”

“…Our previous channel estimation approach [31], [32] is conducted to predict the SNR between AUV and buoy j ∈ {1, . .…”

“…Nevertheless, the communication-aware motion planning algorithms in [28], [29], and [30] are not developed in the context of AUV. In our previous works [31], [32], the communicationaware motion planning algorithms were designed for AUV in obstacle environment, however they adopt the distance-based measurement to perform collision-avoidance maneuvers. Note that the distance-based solution ignores the dimensions of AUV and obstacles, which is not suitable for AUV in obstacledense environment.…”

“…By considering the sizes of obstacle and AUV, it can effectively reflect the threat level of obstacles to AUV. Compared with the distance-based obstacle avoidance algorithms in [10], [11], [31], and [32], the vision-based solution in this paper can improve the collisionavoidance efficiency in path lengths and control efforts.…”