Reliable Water-Air Direct Wireless Communication: Kalman Filter-Assisted Deep Reinforcement Learning Approach

Abstract

Optical wireless communication (OWC) is an emerging technology for direct communication through the water-air interface. However, due to the high directionality of optical beams and the harsh oceanic environment, it faces significant challenges to achieve the alignment and preserve the link availability, as the waves cause beam deflection and the mobility of the transceivers makes the link worse. To tackle these challenges and achieve reliable optical communication between autonomous underwater vehicles and unmanned aerial vehicles, we propose a deep reinforcement learning algorithm assisted by an extended Kalman filter to solve the alignment issue. To improve the reliability of communication, we present an algorithm to obtain the optimal beam divergence angle to maximize the link availability. The numerical simulations demonstrate that the proposed scheme achieves better performance in terms of energy consumption and alignment accuracy, and the link availability is increased by 25% compared to that without adjustment.

High-Speed Underwater Optical Wireless Communication with Advanced Signal Processing Methods Survey

“…Another ML algorithm explored in the UOWC system is reinforcement learning, which focuses on developing an optimized strategy by monitoring how an intelligent agent acts in an environment to maximize cumulative reward. The reinforcement learning method has been applied to improve the communication stability [118][119][120] and to reduce the power consumption and improve link quality via optimizing the routing protocol in an underwater sensor network [121,122]. In Section 4, we provide a comprehensive survey on the recent UOWC progress based on ML algorithms.…”

“…Results show that the success rate for the transmitting AUV to maintain the LOS link for ten time steps was 97.53% after 10,000 episodes in the simulation environment. In [120], a deep reinforcement learning algorithm assisted by an extended Kalman filter was employed to improve the reliability of water-air optical wireless communication between AUVs and unmanned aerial vehicles (UAVs), which is even more challenging compared with connecting AUVs only. Results show that the proposed learning algorithm achieves a shorter MSE (0.02 m) compared with the triangular exploration (TE) algorithm (0.06 m), a shorter flight distance (1.1 m compared to 2 m on TE), and a smoother trajectory (3.23 compared to 6.98 on TE), which implies a higher alignment accuracy and smaller energy consumption.…”

AUV-Aided Optical—Acoustic Hybrid Data Collection Based on Deep Reinforcement Learning

“…Currently, although underwater acoustic communication (UAC) has become the most widely used technology due to its unique advantages (e.g., long-range communication), it is limited by its shortcomings (e.g., low bandwidth, slow speed, high bit error rate and large delay) [ 9 ]. To address these issues, underwater optical communication (UOC) has emerged as an alternative solution, as it has a higher propagation speed (2.255 m/s) and higher data rate (up to hundreds of Mbit/s) over short to medium-range transmissions [ 10 , 11 ]. As both acoustic and optical communication have their pros and cons, employing multi-modal underwater communication systems in UWSNs has become a potential approach to improve network performance [ 12 , 13 ].…”