Deep Reinforcement Learning for Computation Offloading and Resource Allocation in Unmanned-Aerial-Vehicle Assisted Edge Computing

Abstract

Computation offloading technology extends cloud computing to the edge of the access network close to users, bringing many benefits to terminal devices with limited battery and computational resources. Nevertheless, the existing computation offloading approaches are challenging to apply to specific scenarios, such as the dense distribution of end-users and the sparse distribution of network infrastructure. The technological revolution in the unmanned aerial vehicle (UAV) and chip industry has granted UAVs more computing resources and promoted the emergence of UAV-assisted mobile edge computing (MEC) technology, which could be applied to those scenarios. However, in the MEC system with multiple users and multiple servers, making reasonable offloading decisions and allocating system resources is still a severe challenge. This paper studies the offloading decision and resource allocation problem in the UAV-assisted MEC environment with multiple users and servers. To ensure the quality of service for end-users, we set the weighted total cost of delay, energy consumption, and the size of discarded tasks as our optimization objective. We further formulate the joint optimization problem as a Markov decision process and apply the soft actor–critic (SAC) deep reinforcement learning algorithm to optimize the offloading policy. Numerical simulation results show that the offloading policy optimized by our proposed SAC-based dynamic computing offloading (SACDCO) algorithm effectively reduces the delay, energy consumption, and size of discarded tasks for the UAV-assisted MEC system. Compared with the fixed local-UAV scheme in the specific simulation setting, our proposed approach reduces system delay and energy consumption by approximately 50% and 200%, respectively.

Data-Driven Analytics Task Management at the Edge: A Fuzzy Reasoning Approach

“…The amount of data availability in each node has been overlooked in this study. Moreover, Li et al [9] developed a theoretical contract-based offloading paradigm from communication and computing perspectives. The paradigm focuses on computeintensive and delay-sensitive tasks.…”

“…Autonomous driving, smart cities services, and Augmented Reality are just a few examples of new computational-intensive and data-driven applications over the IoT infrastructure [9]. Many of these applications are delay-sensitive and necessitate predictive, analytics and machine learning processes that are thought to be beyond the capability of end-user devices [2].…”

“…In particular, EC has been adopted to support real-time execution and data analytics applications. As a consequence, end-users demand execution and delivery of data-driven tasks from EC [7], [9]. EC has limited capacity making the holistic task execution of all requests a challenge, especially, when these tasks involve 'intense' data processing (e.g., clustering and classification) [6].…”

Time-Constrained Node Visit Planning for Collaborative UAV–WSN Distributed Applications

“…The offloading idea has also been extended to the possibility of forwarding data to an edge server on the ground [ 10 ] instead of performing all computation on board the UAV. In the same scenario, the computation offloading policy has been determined through reinforcement learning with improved results with respect to the strategy of sticking to a single policy [ 11 ].…”

Intelligent Pricing Model for Task Offloading in Unmanned Aerial Vehicle Mounted Mobile Edge Computing for Vehicular Network

“…The results demonstrated that by utilizing sliced edge computing resources, the approach reliably reduced battery energy consumption over a wide range of task complications and task deadlines, prolonging the battery lives of mobile devices. The offloading decision and resource allocation problem were investigated in [19] as a multi-user and multi-server UAV-assisted MEC environment. To ensure the quality of service for end users, they set the weighted total cost of delay, energy consumption, and the size of discarded tasks as the optimization objective.…”