Dynamic sub-route-based self-adaptive beam search Q-learning algorithm for traveling salesman problem

Abstract

In this paper, a dynamic sub-route-based self-adaptive beam search Q-learning (DSRABSQL) algorithm is proposed that provides a reinforcement learning (RL) framework combined with local search to solve the traveling salesman problem (TSP). DSRABSQL builds upon the Q-learning (QL) algorithm. Considering its problems of slow convergence and low accuracy, four strategies within the QL framework are designed first: the weighting function-based reward matrix, the power function-based initial Q-table, a self-adaptive ε-beam search strategy, and a new Q-value update formula. Then, a self-adaptive beam search Q-learning (ABSQL) algorithm is designed. To solve the problem that the sub-route is not fully optimized in the ABSQL algorithm, a dynamic sub-route optimization strategy is introduced outside the QL framework, and then the DSRABSQL algorithm is designed. Experiments are conducted to compare QL, ABSQL, DSRABSQL, our previously proposed variable neighborhood discrete whale optimization algorithm, and two advanced reinforcement learning algorithms. The experimental results show that DSRABSQL significantly outperforms the other algorithms. In addition, two groups of algorithms are designed based on the QL and DSRABSQL algorithms to test the effectiveness of the five strategies. From the experimental results, it can be found that the dynamic sub-route optimization strategy and self-adaptive ε-beam search strategy contribute the most for small-, medium-, and large-scale instances. At the same time, collaboration exists between the four strategies within the QL framework, which increases with the expansion of the instance scale.

A Multi-layer Guidance Approach for Submerged Sensor Networks Integrating Acoustic and Optical Technologies

“…While there has been notable progress in underwater acoustic communications in recent years, these advancements primarily focus on addressing the inherent difficulties of underwater propagation, such as signal attenuation and multipath effects. However, even with these improvements, the long signal propagation delay and limited bandwidth still pose significant obstacles for achieving high-bandwidth video transmission in Underwater Acoustic Sensor Networks (UW-ASNs) [8]. The delays introduced by the relatively slow speed of sound underwater, coupled with the limited data rates achievable through acoustic signals, make the seamless transmission of high-quality video data a complex task.…”