Q-Learning-Based Spectrum Access for Content Delivery in Mobile Networks

Su, Zhou; Dai, Minghui; Xu, Qiuliang; Li, Ruidong; Fu, Shu

doi:10.1109/tccn.2020.2971688

Cited by 18 publications

(11 citation statements)

References 46 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Therefore, reinforcement learning methods are widely used in offloading [18]. Zhou et al [19] proposed a Q-Learning-based spectrum access scheme in mobile networks to enable mobile users to access the optimal spectrum and maximize the transmission rate. Dab et al [9] proposed a QL-JTAR algorithm to minimize the energy consumption of mobile terminals under application delay constraints.…”

Section: Related Workmentioning

confidence: 99%

A Task Offloading Algorithm With Cloud Edge Jointly Load Balance Optimization Based on Deep Reinforcement Learning for Unmanned Surface Vehicles

Yan

Chen

et al. 2022

IEEE Access

View full text Add to dashboard Cite

Unmanned Surface Vehicles (USVs) generate a large amount of data that needs to be processed in real time when they work, but they are usually limited by computational and battery resources, so they need to offload their tasks to the edge for processing. However, when numerous USVs offload their tasks to the edge nodes, some offloaded tasks may be thrown due to queuing timeouts. Existing task offloading methods generally consider the latency or the overall system energy consumption caused by the collaborative processing at the edge and end layers, and do not consider the wasted energy when the tasks are thrown. Therefore, to address the above situation, this paper establishes a task offloading model to minimize long-term task latency and energy consumption by jointly considering the requirments of latency and energy-sensitive tasks and the overall load dynamics in the cloud, edge, and end layers. A deep reinforcement learning (DRL)-based Task Offloading with Cloud Edge Jointly Load Balance Optimization algorithm (TOLBO) is proposed to select the best edge server or cloud server for offloading. Simulation results show that the algorithm can improve the utilization of energy consumption of the cloud edge nodes compared with other algorithms. At the same time, it can significantly reduce the task throw rate, average latency, and energy consumption of end devices. INDEX TERMS USV, mobile edge computing, offloading delay, energy consumption, DRL I. INTRODUCTION

show abstract

Section: Related Workmentioning

confidence: 99%

A Task Offloading Algorithm With Cloud Edge Jointly Load Balance Optimization Based on Deep Reinforcement Learning for Unmanned Surface Vehicles

Yan

Chen

et al. 2022

IEEE Access

View full text Add to dashboard Cite

show abstract

“…In [22], Wang et al transform the edge caching problem as a Markov decision process and propose a distributed cache replacement strategy based on Q-learning to address the optimization problem. In [23], Su et al propose a Q-learning-based spectrum access scheme to optimize spectrum and maximize the transmission rate. In [24], Dinh et al propose a Qlearning-based scheme to solve the optimization problem in a multiuser multiedge-node computation offloading scenario.…”

Section: Related Workmentioning

confidence: 99%

Deep Reinforcement Learning for Collaborative Computation Offloading on Internet of Vehicles

2021

Wireless Communications and Mobile Computing

View full text Add to dashboard Cite

With the increase of Internet of vehicles (IoVs) traffic, the contradiction between a large number of computing tasks and limited computing resources has become increasingly prominent. Although many existing studies have been proposed to solve this problem, their main consideration is to achieve different optimization goals in the case of edge offloading in static scenarios. Since realistic scenarios are complicated and generally time-varying, these studies in static scenes are imperfect. In this paper, we consider a collaborative computation offloading in a time-varying edge-cloud network, and we formulate an optimization problem with considering both delay constraints and resource constraints, aiming to minimize the long-term system cost. Since the set of feasible solutions to the problem is nonconvex, and the complexity of the problem is very large, we propose a Q-learning-based approach to solve the optimization problem. In addition, due to the dimensional catastrophes, we further propose a DQN-based approach to solve the optimization problem. Finally, by comparing our two proposed algorithms with typical algorithms, the simulation results show that our proposed approaches achieve better performance. Under the same conditions, by comparing our two proposed algorithms with typical algorithms, the simulation results show that our proposed Q-learning-based method reduces the system cost by about 49% and 42% compared to typical algorithms. And in the same case, compared with the classical two schemes, our proposed DQN-based scheme reduces the system cost by 62% and 57%.

show abstract

“…2) Caching Assisted Wireless Networks: In recent years, wireless caching has been considered as one of most important techniques to improve QoS of users as well as reduce the costs of wireless network operations, which has attracted more and more research attentions [29]. An increasing number of research contributions have been made in existing works that mainly focus on wireless caching strategies [30]- [35]. In [30], a content-centric transmission mechanism was proposed in a cloud radio access network (C-RAN).…”

Section: Introductionmentioning

confidence: 99%

“…Furthermore, RF WET has been integrated with fog radio access network (F-RAN) to guarantee the downloading requirement of the content users and the wireless charging requirement of the energy users simultaneously [34]. In [35], multiple device-to-device (D2D) pairs provide the contents with caching capacities, where the mobile users access the optimal spectrum to download the contents, and a Q-learning spectrum access scheme is proposed to maximize the transmission rate.…”

Section: Introductionmentioning

confidence: 99%

Intelligent-Reflecting-Surface-Empowered Wireless-Powered Caching Networks

Chu

Xiao

Shojafar

et al. 2022

IEEE Internet Things J.

View full text Add to dashboard Cite

In this paper, we propose an intelligent reflecting surface (IRS) enabled wireless powered caching system. In the proposed IRS model, a power station (PS) provides wireless energy to multiple Internet of Things (IoT) devices, delivering their information to an access point (AP) by utilizing the harvested power. The AP, equipped with a local cache, stores the IoT data to avoid waking up the IoT devices frequently. Meanwhile, we deploy the IRS involving in the wireless energy and information transfer process for performance enhancements. In this practical system, the PS and the AP could belong to different service providers. Also, the AP requires to incentivize the PS to offer a provisional energy service. We model the interaction between the PS and the AP as a Stackelberg game that jointly optimizes the transmit power of the PS, the energy price, the phase shifts of the wireless energy transfer (WET) and wireless information transfer (WIT) phases, as well as wireless caching strategies of the AP. In this way, we first derive the optimal solutions of the phase shifts and the transmit power of the PS in closed-form. We propose an alternating optimization (AO) algorithm to optimize the wireless caching strategies and the energy price iteratively. Finally, we present various numerical evaluations to validate the beneficial role of the IRS and the wireless caching strategies and the performance of the proposed scheme compared with the existing benchmark schemes.

show abstract

Q-Learning-Based Spectrum Access for Content Delivery in Mobile Networks

Cited by 18 publications

References 46 publications

A Task Offloading Algorithm With Cloud Edge Jointly Load Balance Optimization Based on Deep Reinforcement Learning for Unmanned Surface Vehicles

A Task Offloading Algorithm With Cloud Edge Jointly Load Balance Optimization Based on Deep Reinforcement Learning for Unmanned Surface Vehicles

Deep Reinforcement Learning for Collaborative Computation Offloading on Internet of Vehicles

Intelligent-Reflecting-Surface-Empowered Wireless-Powered Caching Networks

Contact Info

Product

Resources

About