Simulated Annealing Based Hierarchical Q-Routing: A Dynamic Routing Protocol

Lopez, Antonio Mira; Heisterkamp, Douglas R.

doi:10.1109/itng.2011.138

Cited by 10 publications

(5 citation statements)

References 8 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To assess the performance of the proposed method, we compare it against the state-of-the-art Q-routing algorithms discussed in section I, including (i) Random Exploration-Exploitation Routing (REE-Routing), (ii) Probabilistic Ex-ploration Routing (PE-Routing), (iii) Conventional [18], (iv) Adaptive learning rates Full-Echo Q-Routing (AFEQ-Routing) [24], and (v) Simulated Annealing based Q-routing (SAHQ-Routing) [54]. Methods (i) and (ii) are simulated for the sake of comparison only.…”

Section: Simulation Resultsmentioning

confidence: 99%

See 1 more Smart Citation

Fully-Echoed Q-Routing With Simulated Annealing Inference for Flying Adhoc Networks

Rovira-Sugranes

Afghah

et al. 2021

IEEE Trans. Netw. Sci. Eng.

View full text Add to dashboard Cite

Current networking protocols deem inefficient in accommodating the two key challenges of Unmanned Aerial Vehicle (UAV) networks, namely the network connectivity loss and energy limitations. One approach to solve these issues is using learning-based routing protocols to make close-tooptimal local decisions by the network nodes, and Q-routing is a bold example of such protocols. However, the performance of the current implementations of Q-routing algorithms is not yet satisfactory, mainly due to the lack of adaptability to continued topology changes. In this paper, we propose a fullecho Q-routing algorithm with a self-adaptive learning rate that utilizes Simulated Annealing (SA) optimization to control the exploration rate of the algorithm through the temperature decline rate, which in turn is regulated by the experienced variation rate of the Q-values. Our results show that our method adapts to the network dynamicity without the need for manual re-initialization at transition points (abrupt network topology changes). Our method exhibits a reduction in the energy consumption ranging from 7% up to 82%, as well as a 2.6 fold gain in successful packet delivery rate, compared to the state of the art Q-routing protocols 1 .

show abstract

Section: Simulation Resultsmentioning

confidence: 99%

“…This gain is achieved by controlling Fig. 7: The average of the temperature parameter T over time for the proposed method, as well as the baseline method with non-adaptive temperature [54] for networks with different average speeds.…”

Section: Simulation Resultsmentioning

confidence: 99%

Fully-Echoed Q-Routing With Simulated Annealing Inference for Flying Adhoc Networks

Rovira-Sugranes

Afghah

et al. 2021

IEEE Trans. Netw. Sci. Eng.

View full text Add to dashboard Cite

show abstract

“…With the decrease of temperature, the global optimal solution is explored randomly in the solution space according to the Metropolis criterion to avoid falling into local optimum. For the routing problem, SA is combined into Q-routing algorithm which has self-adaptive learning rate for dynamic exploration [36,37]. We utilize SA algorithm to dynamically adjust the exploration rate and dynamic greedy factor for RL through the temperature decline rate.…”

Section: Related Workmentioning

confidence: 99%

Reinforcement Learning Based Relay Selection for Underwater Acoustic Cooperative Networks

Zhang

Shen

et al. 2022

Remote Sensing

View full text Add to dashboard Cite

In the complex and dynamically varying underwater acoustic (UWA) channel, cooperative communication can improve throughput for UWA sensor networks. In this paper, we design a reasonable relay selection strategy for efficient cooperation with reinforcement learning (RL), considering the characteristics of UWA channel variation and long transmission delay. The proposed scheme establishes effective state and reward expression to better reveal the relationship between RL and UWA environment. Meanwhile, simulated annealing (SA) algorithm is integrated with RL to improve the performance of relay selection, where exploration rate of RL is dynamically adapted by SA optimization through the temperature decline rate. Furthermore, the fast reinforcement learning (FRL) strategy with pre-training process is proposed for practical UWA network implementation. The whole proposed SA-FRL scheme has been evaluated by both simulation and experimental data. The simulation and experimental results show that the proposed relay selection scheme can converge more quickly than classical RL and random selection with the increase of the number of iterations. The reward, access delay and data rate of SA-FRL can converge at the highest value and are close to the ideal optimum value. All in all, the proposed SA-FRL relay selection scheme can improve the communication efficiency through the selection of the relay nodes with high link quality and low access delay.

show abstract

“…Simulation results show that, PQ-routing is superior to Q-routing in terms of both learning speed and adaptability. There also many other extensions of Q-routing, e.g., full echo Q-routing [220], dual reinforcement Q-routing [221], ant-based Q-Routing [222], gradient ascent Q-routing [223], Q-probabilistic routing [224] and simulated annealing based hierarchical Q-routing [225]. In [142], a self-learning routing protocol based on reinforcement learning (RLSRP), specific for the FANET, is studied.…”

Section: • Intelligent Algorithms Boost the Communication And Networkingmentioning

confidence: 99%

“…The intelligent algorithms could be exploited to improve the performance of the communication and networking from different layers of the network. USL; USoS [142], [210]- [225] Multi-agent decisions reduce the communication overhead: Multi-agent decision can be introduced to let UAVs decide whether to, when to, what to, and whom to communicate. It makes the communication more precise, and the most needed information being transmitted to the most wanted nodes at the most appropriate time can be achieved.…”

Section: Intelligent Algorithms Boost the Communication And Networkingmentioning

confidence: 99%

Survey on Unmanned Aerial Vehicle Networks: A Cyber Physical System Perspective

Wang

Zhao

Zhang

et al. 2020

IEEE Commun. Surv. Tutorials

188

View full text Add to dashboard Cite

Unmanned aerial vehicle (UAV) networks are playing an important role in various areas due to their agility and versatility, which have attracted significant attention from both the academia and industry in recent years. As an integration of the embedded systems with communication devices, computation capabilities and control modules, the UAV network could build a closed loop from data perceiving, information exchanging, decision making to the final execution, which tightly integrates the cyber processes into the physical devices. Therefore, the UAV network could be considered as a cyber physical system (CPS). Revealing the coupling effects among the three interacted components in this CPS system, i.e., communication, computation and control, is envisioned as the key to properly utilize all the available resources and hence improve the performance of the UAV networks. In this paper, we present a comprehensive survey on the UAV networks from a CPS perspective. Firstly, we respectively research the basics and advances with respect to the three CPS components in the UAV networks. Then we look inside to investigate how these components contribute to the system performance by classifying the UAV networks into three hierarchies, i.e., the cell level, the system level, and the system of system level. Further, the coupling effects among these CPS components are explicitly illustrated, which could be enlightening to deal with the challenges in each individual aspect. New research directions and open issues are discussed at the end of this survey. With this intensive literature review, we try to provide a novel insight into the state-of-the-art in the UAV networks.

show abstract

Simulated Annealing Based Hierarchical Q-Routing: A Dynamic Routing Protocol

Cited by 10 publications

References 8 publications

Fully-Echoed Q-Routing With Simulated Annealing Inference for Flying Adhoc Networks

Fully-Echoed Q-Routing With Simulated Annealing Inference for Flying Adhoc Networks

Reinforcement Learning Based Relay Selection for Underwater Acoustic Cooperative Networks

Survey on Unmanned Aerial Vehicle Networks: A Cyber Physical System Perspective

Contact Info

Product

Resources

About