Optimised Q‐learning for WiFi offloading in dense cellular networks

WiFi Offloading Algorithm Based on Q-Learning and MADM in Heterogeneous Networks

“…As can be seen from Figure 3, after filtering out the invalid network whose throughput is less than the threshold V TP th in advance, the convergence speed of the Q-learning can be greatly accelerated. Figures 4 and 5 show the comparison between this paper's algorithm, Fakhfakh and Hamouda's algorithm [11], and RSS (received signal strength) algorithm based on user satisfaction, throughput, power consumption, cost, and delay under stream service. We repeatedly scatter APs 1000 times to eliminate randomness.…”

“…e number of user-passed positions N p is equal to 10, and the number of WiFi AP is changed from 20 to 60. As can be seen from Figure 4, the WiFi offloading algorithm in this paper is superior to the Input: state set S, action set A, paired comparison matrix B, candidate network attribute matrix X, and iteration limit Z Output: trained Q-table, best action selection strategy Π * , and user satisfaction Φ sat j (1) Calculate attribute weights based on B (2) For s ∈ S, a ∈ A (3) Q(s, a) � 0 (4) End For (5) Randomly choose s ini ∈ S as the initialization state (6) While iteration < Z (7) For each state (8) If rand < ε (9) Randomly choose an action (10) Else (11) Select the action corresponding to the maximum Q value in this state. (12) End If (13) Perform a (14) Calculate Rw t (s, a) according to equation (23) (15) Observe the next state s′ (16) Update the Q- Mobile Information Systems other two algorithms in user satisfaction.…”

“…Q-learning consists of three elements, including state, action, and reward. e state set is denoted as S and the action set is denoted as A, and the purpose of Q-learning is to obtain the optimal action selection strategy Π * to maximize the agent's discounted cumulative reward [11]. In state s ∈ S, the agent selects an action a ∈ A from the action set to act on the environment.…”

“…According to the above references, the most challenging problem in WiFi offloading is how to make an offloading decision, that is, how to choose the most suitable WiFi AP for communication. Fakhfakh and Hamouda [11] aimed to minimize the residence time of the cellular network and optimized it by Q-learning. e reward function considers SINR, handover delay, and AP load.…”

Reinforcement learning-based dynamic band and channel selection in cognitive radio ad-hoc networks

“…Rashed et al studied the reinforcement learning that maximizes the sum-rate of D2D users and cellular users to minimize the interference in a D2D environment [21]. Fakhfakh and Hamouda used the received SINR from the access point (AP) detected by the mobile user, QoS metrics about the channel load, and delay as the reward for choosing a WiFi over a cellular network to apply WiFi offloading and reducing the load on the cellular network [22]. Yan et al propose a smart aggregated radio access technologies (RAT) access strategy with the aim of maximizing the long-term network throughput while meeting diverse traffic quality of service requirements by using Q-learning [23].…”