Unmanned-Aerial-Vehicle-Assisted Computation Offloading for Mobile Edge Computing Based on Deep Reinforcement Learning

Wang, Hui; Ke, Hongchang; Sun, Weijia

doi:10.1109/access.2020.3028553

Cited by 41 publications

(21 citation statements)

References 36 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…2) Mobile MEC Server: To satisfy the extensive service requests of a tremendous number of mobile devices, vehicle-and UAV-aided network architectures have been proposed with mobile MEC servers [60]- [64], [76], [88], [95], [96], [133]- [139]. Due to the flexible coverage of the movable MEC servers, the computational service range is sufficiently extended.…”

Section: Studies On Distributed Service 1) Complex Service Deploymentmentioning

confidence: 99%

“…To minimize the total time and energy consumption of all mobile devices, Q-learning-based task offloading and resource allocation were proposed in [135]. Furthermore, considering the renewable power supply for a UAV with stochastic task arrival through the time-varying channel, a model-free DRLbased computation offloading policy was proposed in [76],…”

Section: Studies On Distributed Service 1) Complex Service Deploymentmentioning

confidence: 99%

See 1 more Smart Citation

Reinforcement Learning-Empowered Mobile Edge Computing for 6G Edge Intelligence

Wei,

Guo,

et al. 2022

Preprint

View full text Add to dashboard Cite

Mobile edge computing (MEC) is considered a novel paradigm for computation-intensive and delay-sensitive tasks in fifth generation (5G) networks and beyond. However, its uncertainty, referred to as dynamic and randomness, from the mobile device, wireless channel, and edge network sides, results in high-dimensional, nonconvex, nonlinear, and NP-hard optimization problems. Thanks to the evolved reinforcement learning (RL), upon iteratively interacting with the dynamic and random environment, its trained agent can intelligently obtain the optimal policy in MEC. Furthermore, its evolved versions, such as deep RL (DRL), can achieve higher convergence speed efficiency and learning accuracy based on the parametric approximation for the large-scale state-action space. This paper provides a comprehensive research review on RL-enabled MEC and offers insight for development in this area. More importantly, associated with free mobility, dynamic channels, and distributed services, the MEC challenges that can be solved by different kinds of RL algorithms are identified, followed by how they can be solved by RL solutions in diverse mobile applications. Finally, the open challenges are discussed to provide helpful guidance for future research in RL training and learning MEC. Index Terms-Mobile edge computing (MEC), network uncertainty, reinforcement learning (RL). emote Areas irplane taverse vers Diverse Scenarios MEC Deployment Future Applications Smart City Smart Home Hazard/Remote Areas Smart Agriculture Digital Twin Holographic Communication AR/VR/XR Metaverse Metaverse Metavers Autonomous Driving Small Data Center Switch BS Traffic Light Satellite Ship Airplane S ll S it h BS T ffi Li ht S t llit Shi Ai l 5G Networks and Beyond 5G Networks and Beyond Smart Factory performance. Its typical algorithms are K-means clustering, principal component analysis, and independent component analysis, which can be used in small cell clustering, heterogeneous network clustering, smart grid user classification, etc. However, in unreliable radio network environments, the classification accuracy decreases, which readily causes slow and inaccurate actions in MEC systems. • Different from the static solutions provided by supervised learning and unsupervised learning, RL gives a constantly evolving intelligent framework [21]-[27]. In RL [29], an agent is enabled to make proper decisions based on frequent interactions with the stochastic and dynamic environment. Based on the Markov models, the RL operates in a feedback mechanism (a closed loop) without the knowledge of input data, where based on the previous and current states, the agent can execute its actions to maximize the reward function. Additionally, the above behaviors, including the states, actions, and rewards, accumulate to generate experiences. The classic RL algorithm is Q-learning, which suffers from the curse of dimensionality caused by the large dimensions of the state-action spaces. Accordingly, upon leveraging the low-dimensional representation for the high-dimensional state-ac...

show abstract

Section: Studies On Distributed Service 1) Complex Service Deploymentmentioning

confidence: 99%

Section: Studies On Distributed Service 1) Complex Service Deploymentmentioning

confidence: 99%

Reinforcement Learning-Empowered Mobile Edge Computing for 6G Edge Intelligence

Wei,

Guo,

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…There exist works on resource management with different architectures including {cloud, edge servers, and drones [5], [7], [27]}, {mobile users, drones [28]}, {wireless users, edge servers, and drones [29]}, {drones and edge servers [30], [31]}, and {drones, edge servers, and smart mobile devices [32], [33]}.…”

Section: A Resource Management In a Group Of Drones In Different Architecturesmentioning

confidence: 99%

Fast and Fair Computation Offloading Management in a Swarm of Drones Using a Rating-Based Federated Learning Approach

et al. 2021

View full text Add to dashboard Cite

Today, unmanned aerial vehicles (UAVs) or drones are increasingly used to enable and support multi-access edge computing (MEC). However, transferring data between nodes in such dynamic networks implies considerable latency and energy consumption, which are significant issues for practical real-time applications. In this paper, we consider an autonomous swarm of heterogeneous drones. This is a general architecture that can be applied for applications that need in-field computation, e.g. real-time object detection in video streams. Collaborative computing in a swarm of drones has the potential to improve resource utilization in a real-time application i.e., each drone can execute computations locally or offload them to other drones. In such an approach, drones need to compete for using each other's resources; therefore, efficient orchestration of the communication and offloading at the swarm level is essential. The main problem investigated in this work is computation offloading between drones in a swarm. To tackle this problem, we propose a novel federated learning (FL)-based fast and fair offloading strategy with a rating method. The simulation results demonstrate the effectiveness of the proposed strategy over other existing methods and architectures with average improvements of -23% in energy consumption, -15% in latency, +18% in throughput, and +9% in fairness.INDEX TERMS Swarm of drones, Multi-access edge computing, Collaborative computing, Federated learning.

show abstract

“…The optimal latency is achieved by managing the spectrum, computation, and caching resources. Besides that, the scenarios of unstable energy arrival, stochastic computation tasks from VUEs, and time-varying channel state are analyzed to reduce latency [98]. UAVs are dynamically deployed to support the computation offloading.…”

Section: Drl-based Techniquesmentioning

confidence: 99%

DRL‐Based Intelligent Resource Allocation for Diverse QoS in 5G and toward 6G Vehicular Networks: A Comprehensive Survey

Nguyen

et al. 2021

Wireless Communications and Mobile Computing

View full text Add to dashboard Cite

The vehicular network is taking great attention from both academia and industry to enable the intelligent transportation system (ITS), autonomous driving, and smart cities. The system provides extremely dynamic features due to the fast mobile characteristics. While the number of different applications in the vehicular network is growing fast, the quality of service (QoS) in the 5G vehicular network becomes diverse. One of the most stringent requirements in the vehicular network is a safety-critical real-time system. To guarantee low-latency and other diverse QoS requirements, wireless network resources should be effectively utilized and allocated among vehicles, such as computation power in cloud, fog, and edge servers; spectrum at roadside units (RSUs); and base stations (BSs). Historically, optimization problems have mostly been investigated to formulate resource allocation and are solved by mathematical computation methods. However, the optimization problems are usually nonconvex and hard to be solved. Recently, machine learning (ML) is a powerful technique to cope with the complexity in computation and has capability to cope with big data and data analysis in the heterogeneous vehicular network. In this paper, an overview of resource allocation in the 5G vehicular network is represented with the support of traditional optimization and advanced ML approaches, especially a deep reinforcement learning (DRL) method. In addition, a federated deep reinforcement learning- (FDRL-) based vehicular communication is proposed. The challenges, open issues, and future research directions for 5G and toward 6G vehicular networks, are discussed. A multiaccess edge computing assisted by network slicing and a distributed federated learning (FL) technique is analyzed. A FDRL-based UAV-assisted vehicular communication is discussed to point out the future research directions for the networks.

show abstract

Unmanned-Aerial-Vehicle-Assisted Computation Offloading for Mobile Edge Computing Based on Deep Reinforcement Learning

Cited by 41 publications

References 36 publications

Reinforcement Learning-Empowered Mobile Edge Computing for 6G Edge Intelligence

Reinforcement Learning-Empowered Mobile Edge Computing for 6G Edge Intelligence

Fast and Fair Computation Offloading Management in a Swarm of Drones Using a Rating-Based Federated Learning Approach

DRL‐Based Intelligent Resource Allocation for Diverse QoS in 5G and toward 6G Vehicular Networks: A Comprehensive Survey

Contact Info

Product

Resources

About