Reinforcement Learning Based Relay Selection for Underwater Acoustic Cooperative Networks

Zhang, Yuzhi; Su, Yue; Shen, Xiaohong; Wang, Anyi; Wang, Bin; Liu, Yang; Bai, Weigang

doi:10.3390/rs14061417

Cited by 13 publications

(5 citation statements)

References 47 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The cooperative node is used to provide relay assistance to enhance the signal gain of receiving nodes and achieve reliable communication. Zhang et al proposed SA-FRL [35] to realize the cooperative communication of underwater networks based on Q-learning. SA-FRL selects cooperative nodes with good link quality and low access delay to improve the efficiency of cooperative communication.…”

Section: Related Workmentioning

confidence: 99%

“…The V-value of the not-selected candidate cooperative node nodes is not updated. With the iteration of training, the Vvalue will be continuously updated according to (34) and (35) and gradually converge, and a good cooperative communication policy will finally be highlighted. Figure 4 shows how the policy is generated based on the cooperative communication sub-algorithm.…”

Section: Cooperative Communication Sub-algorithmmentioning

confidence: 99%

See 1 more Smart Citation

SQMCR: Stackelberg Q-Learning-Based Multi-Hop Cooperative Routing Algorithm for Underwater Wireless Sensor Networks

Bin,

Kerong,

Yixue

et al. 2024

IEEE Access

View full text Add to dashboard Cite

The underwater wireless sensor network (UWSNs) is an important communication facility supporting underwater monitoring applications. However, the transmission channel has the characteristics of high bit error rate, strong multipath effect, and many interference factors, and the network node has the characteristics of high energy consumption, difficult energy supply, and the node position vulnerable to change, which makes it extremely difficult for UWSNs to realize the reliable and efficient packet forwarding. To address the problem, we propose the Stackelberg Q-learning based multi-hop cooperative routing algorithm (SQMCR). The SQMCR builds the transmission routes based on the Q-learning algorithm, considering factors such as the delay, the remaining energy, and the network topology, which improves the rationality and adaptability of selecting the next-hop node. By balancing the packet forwarding benefits and the energy consumption costs based on the Stackelberg Q-learning algorithm, the SQMCR establishes the cooperative communication policy to ensure both the reliability and efficiency of underwater communications. It also adopts initializing Q-values and dynamic exploration probabilities optimization methods to further improve the performance of routing algorithms. Experimental results show that the SQMCR can help UWSNs increase the packet forwarding reliability and prolong the network lifetime by 17%. It has a better environment and application adaptability and is more suitable for underwater high-reliability applications.INDEX TERMS underwater wireless sensor networks (UWSNs), routing algorithm, cooperative communication, Q-learning, Stackelberg game. I. INTRODUCTION U NDERWATER wireless sensor networks are an important part of the construction of the marine Internet of Things [1] and an important part of the underwater direction of the future 6G network [2]. They are widely used in many fields, such as disaster early warning, pollutant monitoring, hydrological data monitoring, marine resource exploration, auxiliary navigation, and as an important infrastructure for studying, building, and developing the ocean [3]. Underwater wireless sensor networks are composed of sensor nodes, communication nodes, and sink nodes [4]. At present, the long distance underwater wireless transmission of data mainly depends on the acoustic channel [5]. The underwater acoustic channel has many problems, such as large transmission delay, limited transmission bandwidth, many interference factors, and serious multipath phenomena [6], [7]. Underwater communication nodes are affected by water flow, and their positions and node relationships change dynamically, their communication energy consumption is high, and the energy supplies for the nodes are difficult [6], [7], [8]. All the unfavorable factors make reliable underwater communication extremely difficult. But the reliable communication is the base of various applications in underwater networks [9]. The reliable communication in underwater wireless sensor networks is reflected not ...

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Cooperative Communication Sub-algorithmmentioning

confidence: 99%

SQMCR: Stackelberg Q-Learning-Based Multi-Hop Cooperative Routing Algorithm for Underwater Wireless Sensor Networks

Bin,

Kerong,

Yixue

et al. 2024

IEEE Access

View full text Add to dashboard Cite

show abstract

“…It introduces a recovery mechanism to minimize the impact of routing voids on the data transmission performance. Zhang et al [27] introduced a reinforcement learning-based relay selection algorithm for UWSNs, combining RL with a simulated annealing algorithm to enhance the algorithm's performance. Ye et al [28] suggested a deep reinforcement learning-based medium access control protocol for underwater acoustic networks.…”

Section: Related Workmentioning

confidence: 99%

RL-ANC: Reinforcement Learning-Based Adaptive Network Coding in the Ocean Mobile Internet of Things

Zhang,

Wang

2024

JMSE

View full text Add to dashboard Cite

As the demand for sensing and monitoring the marine environment increases, the Ocean Mobile Internet of Things (OM-IoT) has gradually attracted the interest of researchers. However, the unreliability of communication links represents a significant challenge to data transmission in the OM-IoT, given the complex and dynamic nature of the marine environment, the mobility of nodes, and other factors. Consequently, it is necessary to enhance the reliability of underwater data transmission. To address this issue, this paper proposes a reinforcement learning-based adaptive network coding (RL-ANC) approach. Firstly, the channel conditions are estimated based on the reception acknowledgment, and a feedback-independent decoding state estimation method is proposed. Secondly, the sliding coding window is dynamically adjusted based on the estimates of the channel erasure probability and decoding probability, and the sliding rule is adaptively determined using a reinforcement learning algorithm and an enhanced greedy strategy. Subsequently, an adaptive optimization method for coding coefficients based on reinforcement learning is proposed to enhance the reliability of the underwater data transmission and underwater network coding while reducing the redundancy in the coding. Finally, the sampling period and time slot table are updated using the enhanced simulated annealing algorithm to optimize the accuracy and timeliness of the channel estimation. Simulation experiments demonstrate that the proposed method effectively enhances the data transmission reliability in unreliable communication links, improves the performance of underwater network coding in terms of the packet delivery rate, retransmission, and redundancy transmission ratios, and accelerates the convergence speed of the decoding probability.

show abstract

“…Utilizing average stochastic-inclination drop (SGD) strategy to prepare a repetitive organization, the backspread blunder signals will generally zero that suggests a restrictively lengthy intermingling time [35][36][37][38][39][40]. To handle this inclination disappearing issue, Hochreiter and Schmidhuber proposed long short-term memory (LSTM) in their trailblazer work of reference [26], which brought cell and entryway into the RNN structure.…”

Section: System Modelmentioning

confidence: 99%

Cooperative Communications Based on Deep Learning Using a Recurrent Neural Network in Wireless Communication Networks

Rathika

Poruran

Kumar

et al. 2022

Mathematical Problems in Engineering

View full text Add to dashboard Cite

In recent years, cooperative communication (CC) technology has emerged as a hotspot for testing wireless communication networks (WCNs), and it will play an important role in the spectrum utilization of future wireless communication systems. Instead of running node transmissions at full capacity, this design will distribute available paths across multiple relay nodes to increase the overall throughput. The modeling WCNs coordination processes, as a recurrent mechanism and recommending a deep learning-based transfer choice, propose a recurrent neural network (RNN) process-based relay selection in this research article. This network is trained according to the joint receiver and transmitter outage likelihood and shared knowledge, and without the use of a model or prior data, the best relay is picked from a set of relay nodes. In this study, we make use of the RNN to do superdimensional (high-layered) processing and increase the rate of learning and also have a neural network (NN) selection testing to study the communication device, find out whether or not it can be used, find out how much the system is capable of, and look at how much energy the network needs. In these simulations, it has been shown that the RNN scheme is more effective on these targets and allows the design to keep converged over a longer period of time. We will compare the accuracy and efficiency of our RNN processed-based relay selection methods with long short-term memory (LSTM), gated recurrent units (GRU), and bidirectional long short-term memory (BLSTM),which are all acronyms for long short-term memory methods.

show abstract

Reinforcement Learning Based Relay Selection for Underwater Acoustic Cooperative Networks

Cited by 13 publications

References 47 publications

SQMCR: Stackelberg Q-Learning-Based Multi-Hop Cooperative Routing Algorithm for Underwater Wireless Sensor Networks

SQMCR: Stackelberg Q-Learning-Based Multi-Hop Cooperative Routing Algorithm for Underwater Wireless Sensor Networks

RL-ANC: Reinforcement Learning-Based Adaptive Network Coding in the Ocean Mobile Internet of Things

Cooperative Communications Based on Deep Learning Using a Recurrent Neural Network in Wireless Communication Networks

Contact Info

Product

Resources

About