Adaptive modulation switching strategy based on Q-learning for underwater acoustic communication channel

Lin, Jiamin; Su, Wei; Xiao, Liang; Jiang, Xialin

doi:10.1145/3291940.3291976

Cited by 8 publications

(7 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The algorithm uses partial, delayed, and noisy state information as well as reward signals for learning. Lin et al (2018) proposed a novel Q-learning-based adaptive modulation-switching strategy to select the appropriate strategy. Landgren et al (2021) designed a dynamic, consensus-based, distributed estimation algorithm.…”

Section: Related Workmentioning

confidence: 99%

ASTS: Autonomous switching of task-level strategies

Wang,

Lv,

Wang

et al. 2023

International Journal of Applied Mathematics and Computer Scien

View full text Add to dashboard Cite

Autonomous coordination of multi-agent systems can improve the reaction and dispatching ability of multiple agents to emergency events. The existing research has mainly focused on the reactions or dispatching in specific scenarios. However, task-level coordination has not received significant attention. This study proposes a framework for autonomous switching of task-level strategies (ASTS), which can automatically switch strategies according to different scenarios in the task execution process. The framework is based on the blackboard system, which takes the form of an instance as an agent and the form of norm(s) as a strategy; it uses events to drive autonomous cooperation among multiple agents. A norm may be triggered when an event occurs. After the triggered norm is executed, it can change the data, state, and event in ASTS. To demonstrate the autonomy and switchability of the proposed framework, we develop a fire emergency reaction dispatch system. This system is applied to emergency scenarios involving fires. Five types of strategies and two control modes are designed for this system. Experiments show that this system can autonomously switch between different strategies and control modes in different scenarios with promising results. Our framework improves the adaptability and flexibility of multiple agents in an open environment and represents a solid step toward switching strategies at the task level.

show abstract

Section: Related Workmentioning

confidence: 99%

ASTS: Autonomous switching of task-level strategies

Wang,

Lv,

Wang

et al. 2023

International Journal of Applied Mathematics and Computer Scien

View full text Add to dashboard Cite

show abstract

“…In addition, [22], [30] predicted MCS values by using boosted tree; [30] shows 99% accuracy. In [29], the cluster algorithm is used, and in [27], [28] MCS are predicted by Q-learning; that one of the reinforcement learning. However, all previous studies have used machine…”

Section: Previous Studiesmentioning

confidence: 99%

Iterative Learning for Reliable Link Adaptation in the Internet of Underwater Things

Byun

Cho

et al. 2021

IEEE Access

View full text Add to dashboard Cite

Given the ever-increasing interest in the Internet of Underwater Things (IoUT), various studies are ongoing to solve some of the practical problems affecting the development of underwater wireless communication. The main problems are related to the use of acoustic waves in a water medium, in which extremely high propagation loss and drastic channel fluctuation are common. On the basis of hands-on experience and measurements made in real underwater environments, the conventional Adaptive Modulation and Coding (AMC), which uses the high correlation between SNR (Signal to Noise Ratio) and BER (Bit Error Rate), might not be affordable in underwater environments because the normal correlation between SNR and BER almost disappears altogether. This work therefore collectively takes into account multiple quality factors of communication at the same time by creating, analysing and validating the machine learning model to predict the most adequate communication parameters to solve the problem. The dataset of underwater wireless communication used in the learning models was obtained from measurements made in a real underwater environment near the Gulf of Incheon, South Korea, using a practical testbed designed and implemented by the authors. The estimated network throughput based on the communications parameters predicted using the machine learning models was enhanced by up to 25% compared with the conventional handcraft method.

show abstract

“…There are a few pioneering studies dealing with the switching issue and providing precious investigations of underwater multimodal wireless networks [13,[27][28][29]. In [13], the authors explored the multimodal switching issue to maximize instantaneous network throughput using the range-based triggering mechanism to proactively switch among different physical layers (PHYs).…”

Section: Introductionmentioning

confidence: 99%

“…Nevertheless, an effective switching strategy should be adaptive to the environmental dynamics, and as reinforcement learning (RL) has the capability of interacting with the environment and could gradually learn an optimal or near-optimal action policy, it is a promising artificial intelligence (AI) tool to develop switching strategy for underwater sensor networks. In a recent research, a model-free RL method is adopted to deal with the dynamics of the channels in order to smoothly switch among different types of acoustic modems in an adaptive manner [28,29]. However, the switching policy in a hybrid optical-acoustic communication system, involving two different types of links, has not been studied much.…”

Section: Introductionmentioning

confidence: 99%

Reinforcement Learning-Based Adaptive Switching Scheme for Hybrid Optical-Acoustic AUV Mobile Network

Luo

Wang

et al. 2022

Wireless Communications and Mobile Computing

View full text Add to dashboard Cite

In an autonomous underwater vehicles– (AUVs–) based optical-acoustic hybrid network, it is critical to achieve ultra high-speed reliable communications, in order to reap the benefits of the complementary systems and perform high-bandwidth and low-latency operations. However, as the mobile AUVs operate in harsh oceanic environments, it is essential to design an effective switching algorithm to execute flexible hybrid acoustic-optical communications and increase the network throughput. In this paper, we propose a Q-learning-based adaptive switching scheme to maximize the network throughput by capturing the dynamics of the varying channels as well as the mobility of AUVs. In order to address the challenge associated with partial observations of the optical channel and improve the switching efficiency in extreme conditions, a blind optical channel estimation method is designed and implemented with the Extended Kalman Filter (EKF), in which the relationship between the underwater acoustic and optical channels is utilized to improve the channel prediction accuracy. Based on this environmental status, a reinforcement learning approach is leveraged to build a near-optimal switching strategy for the hybrid network. We conduct numerical simulations to verify the performance of the scheme, and the simulation results demonstrate that the proposed switching scheme is effective and robust.

show abstract

Adaptive modulation switching strategy based on Q-learning for underwater acoustic communication channel

Cited by 8 publications

References 16 publications

ASTS: Autonomous switching of task-level strategies

ASTS: Autonomous switching of task-level strategies

Iterative Learning for Reliable Link Adaptation in the Internet of Underwater Things

Reinforcement Learning-Based Adaptive Switching Scheme for Hybrid Optical-Acoustic AUV Mobile Network

Contact Info

Product

Resources

About