A Reinforcement Learning based Computing Offloading and Resource Allocation Scheme in F-RAN

Jiang, Fan; Gao, Youjun; Gu, Zesheng

doi:10.21203/rs.3.rs-483062/v1

Cited by 3 publications

(2 citation statements)

References 30 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This indicates that the dueling DQN algorithm might achieve a higher performance than the DQN method when it comes to handling problems related to offloading policies and resources. The work in [114] adopts a dueling DQN algorithm to choose the most suitable offloading strategy for each user equipment (UE), which includes offloading to fog access points (FAP), proximity idle UEs, or processing by itself. As the number of UEs requiring offloading increases, the centralized dueling DQN algorithm becomes more complex.…”

Section: ) Dueling Dqnmentioning

confidence: 99%

Offloading Mechanisms Based on Reinforcement Learning and Deep Learning Algorithms in the Fog Computing Environment

Abdulazeez

Askar

2023

IEEE Access

View full text Add to dashboard Cite

Fog computing has emerged as a computing paradigm for resource-restricted Internet of things (IoT) devices to support time-sensitive and computationally intensive applications. Offloading can be utilized to transfer resource-intensive tasks from resource-limited end devices to a resource-rich fog or cloud layer to reduce end-to-end latency and enhance the performance of the system. However, this advantage is still challenging to achieve in systems with a high request rate because it leads to long queues of tasks in fog nodes and reveals inefficiencies in terms of delays. In this regard, reinforcement learning (RL) is a well-known method for addressing such decision-making issues. However, in large-scale wireless networks, both action and state spaces are complex and extremely extensive. Consequently, reinforcement learning techniques may not be able to identify an efficient strategy within an acceptable time frame. Hence, deep reinforcement learning (DRL) was developed to integrate RL and deep learning (DL) to address this problem. This paper presents a systematic analysis of using RL or DRL algorithms to address offloadingrelated issues in fog computing. First, the taxonomy of fog computing offloading mechanisms based on RL and DRL algorithms was divided into three major categories: value-based, policy-based, and hybridbased algorithms. These categories were then compared based on important features, including offloading problem formulation, utilized techniques, performance metrics, evaluation tools, case studies, their strengths and drawbacks, offloading directions, offloading mode, SDN-based architecture, and offloading decisions. Finally, the future research directions and open issues are discussed thoroughly.INDEX TERMS Fog computing, Internet of Things (IoT), offloading, reinforcement learning, deep reinforcement learning.

show abstract

Section: ) Dueling Dqnmentioning

confidence: 99%

Offloading Mechanisms Based on Reinforcement Learning and Deep Learning Algorithms in the Fog Computing Environment

Abdulazeez

Askar

2023

IEEE Access

View full text Add to dashboard Cite

show abstract

“…DQN may suffer from overestimation due to choosing the maximum action value each time in order to avoid this situation, Double DQN are generated, in [33], delay constraints and uncertain resource requirements of heterogeneous computing tasks, and in order to avoid dimensional disasters and overestimation using DDQN-based algorithms, which eventually proved the effectiveness of the algorithm. It is also possible to change the network structure of DQN to produce Dueling DQN, [34] firstly preprocesses the data and then uses Dueling DQN for optimization of the objective function, its convergence speed is faster than DQN.…”

Section: Related Workmentioning

confidence: 99%

A Vehicle-Assisted Computation Offloading Algorithm Based on Proximal Policy Optimization in Vehicle Edge Networks

Chen

Zeng

et al. 2022

Mobile Netw Appl

View full text Add to dashboard Cite

With the continuous development of the Internet of Vehicles(IoV), Vehicle Edge Computing(VEC) has become a key technology for computational resource scheduling, but more and more smart devices are connected to the internet, which makes it difficult for traditional Vehicle Edge Networks(VEN) to deal with tasks in time. In this paper, in order to cope with the challenges of the large number of devices accessing the internet, we propose a vehicle-assisted computation offloading algorithm based on proximal policy optimization(VCOPPO) for User Equipment(UE) tasks, and it combines dynamic parked vehicles incentives mechanism and computational resource allocation strategy by using road vehicles and parked vehicles as edge servers. Firstly, a non-convex optimization problem combining VEN utility and task processing delay is formulated, subject to the constraints of the residual energy and the transmission rate of the task. Secondly, the proposed VCOPPO is used to solve the formulated non-convex optimization problem, and we use stochastic policy to obtain the optimal computation offloading decisions and resource allocation schemes. Finally, the experimental results have shown that the proposed VCOPPO has an excellent performance in network reward and task processing delay respectively, and it can effectively schedule and allocate computational resources. Compared with using Dueling Deep Q Network(Dueling DQN), Deep Q Network(DQN) and Q-learning methods, the proposed VCOPPO improves the network reward by 31%, 18% and 91%, reduces the delay in task processing by 78%, 63% and 74%, respectively.

show abstract

Reinforcement Learning Methods for Computation Offloading: A Systematic Review

Zabihi,

Eftekhari Moghadam,

Rezvani

2023

ACM Comput. Surv.

View full text Add to dashboard Cite

Today, cloud computation offloading may not be an appropriate solution for delay-sensitive applications due to the long distance between end-devices and remote datacenters. In addition, offloading to a remote cloud can consume bandwidth and dramatically increase costs. On the other hand, end-devices such as sensors, cameras, and smartphones have limited computing and storage capacity. Processing tasks on such battery-powered and energy-constrained devices becomes even more complex. To address these challenges, a new paradigm called Edge Computing (EC) emerged nearly a decade ago to bring computing resources closer to end-devices. Here, edge servers located between the end-device and the remote cloud perform user tasks. Recently, several new computing paradigms such as Mobile Edge Computing (MEC) and Fog Computing (FC) have emerged to complement Cloud Computing (CC) and EC. Although these paradigms are heterogeneous, they can further reduce energy consumption and task response time, especially for delay-sensitive applications. The computation offloading is a multi-objective, NP-hard optimization problem. A significant part of previous research in this field is devoted to Machine Learning (ML) methods. One of the essential types of ML is Reinforcement Learning (RL), in which an agent learns how to make the best decision using the experiences gained from the environment. This paper provides a systematic review of the widely used RL approaches in computation offloading. It covers research in complementary paradigms such as mobile cloud computing (MCC), edge computing, fog computing, and the Internet of Things (IoT). We explain the reasons for using various RL methods in computation offloading from a technical point of view. This analysis includes both binary offloading and partial offloading techniques. For each method, the essential elements of RL and the characteristics of the environment are discussed regarding the most important criteria. Research challenges and Future trends are also mentioned.

show abstract

A Reinforcement Learning based Computing Offloading and Resource Allocation Scheme in F-RAN

Cited by 3 publications

References 30 publications

Offloading Mechanisms Based on Reinforcement Learning and Deep Learning Algorithms in the Fog Computing Environment

Offloading Mechanisms Based on Reinforcement Learning and Deep Learning Algorithms in the Fog Computing Environment

A Vehicle-Assisted Computation Offloading Algorithm Based on Proximal Policy Optimization in Vehicle Edge Networks

Reinforcement Learning Methods for Computation Offloading: A Systematic Review

Contact Info

Product

Resources

About