Deep Reinforcement Learning Based Dynamic Trajectory Control for UAV-Assisted Mobile Edge Computing

Wang, Liang; Wang, Kezhi; Pan, Cunhua; Xu, Wei; Aslam, Nauman; Nallanathan, Arumugam

doi:10.1109/tmc.2021.3059691

Cited by 158 publications

(76 citation statements)

References 37 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In [18], a hierarchical DRL algorithm was developed to minimize the average delay of tasks by jointly optimizing the movement locations of SDs and offloading decisions. To minimize the energy consumption of all SDs, Wang et al [19] presented a trajectory control method based on DDPG with prioritized experience replay. Dai et al [20] considered a UAV-and-BS enabled MEC system and devised a DDPGbased task association scheduling method to minimize the system's energy consumption.…”

Section: Single-objective Optimizationmentioning

confidence: 99%

See 1 more Smart Citation

Evolutionary Multi-Objective Reinforcement Learning Based Trajectory Control and Task Offloading in UAV-Assisted Mobile Edge Computing

Song¹,

Xing²,

Wang³

et al. 2022

Preprint

View full text Add to dashboard Cite

This paper studies the trajectory control and task offloading (TCTO) problem in an unmanned aerial vehicle (UAV)-assisted mobile edge computing system, where a UAV flies along a planned trajectory to collect computation tasks from smart devices (SDs). We consider a scenario that SDs are not directly connected by the base station (BS) and the UAV has two roles to play: MEC server or wireless relay. The UAV makes task offloading decisions online, in which the collected tasks can be executed locally on the UAV or offloaded to the BS for remote processing. The TCTO problem involves multi-objective optimization as its objectives are to minimize the task delay and the UAV's energy consumption, and maximize the number of tasks collected by the UAV, simultaneously. This problem is challenging because the three objectives conflict with each other. The existing reinforcement learning (RL) algorithms, either single-objective RLs or single-policy multi-objective RLs, cannot well address the problem since they cannot output multiple policies for various preferences (i.e. weights) across objectives in a single run. This paper adapts the evolutionary multi-objective RL (EMORL), a multi-policy multi-objective RL, to the TCTO problem. This algorithm can output multiple optimal policies in just one run, each optimizing a certain preference. The simulation results demonstrate that the proposed algorithm can obtain more excellent nondominated policies by striking a balance between the three objectives regarding policy quality, compared with two evolutionary and two multi-policy RL algorithms.

show abstract

Section: Single-objective Optimizationmentioning

confidence: 99%

“…Similar to previous studies [19], [31], we adopt the Cartesian coordinate system to model the movement of the UAV.…”

Section: Uav Movement Modelmentioning

confidence: 99%

Evolutionary Multi-Objective Reinforcement Learning Based Trajectory Control and Task Offloading in UAV-Assisted Mobile Edge Computing

Song¹,

Xing²,

Wang³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…To guarantee global load balance at the UAVs in [22], a problem of minimum global load balance deployment is formulated by jointly optimizing task scheduling and deployment of UAVs under coverage constraints. Deep Reinforcement Learning (DRL) [72], [73] based task scheduling scheme is used for efficient task execution, effective scheduling of offloaded tasks in multi-UAVs, reducing the transmission delay, and improves the QoS of the users.…”

Section: ) Load Balancing and Secrecy Capacitymentioning

confidence: 99%

Energy Efficient UAV-Enabled Mobile Edge Computing for IoT Devices: A Review

Abrar

Ajmal

Almohaimeed³

et al. 2021

IEEE Access

View full text Add to dashboard Cite

With the emergence of computation-intensive and delay-sensitive applications, such as face recognition, virtual reality, augmented reality, and Internet of Things (IoT) devices ; Mobile Edge Computing (MEC) allows the IoT devices to offload their heavy computation tasks to nearby edge cloud network rather than to compute the tasks locally. Therefore, it helps to reduce the energy consumption and execution delay in the ground mobile users. Flying Unmanned Aerial Vehicles (UAVs) integrated with the MEC server play a key role in 5G and future wireless communication networks to provide spatial coverage and further computational services to the small, battery-powered and energy-constrained devices. The UAVenabled MEC (U-MEC) system has flexible mobility and more computational capability compared to the terrestrial MEC network. They support line-of-sight (LoS) links with the users offloading their tasks to the UAVs. Hence, users can transmit more data without interference by mitigating small-scale fading and shadowing effects. UAVs resources and flight time are very limited due to size, weight, and power (SWaP) constraints. Therefore, energy-aware communication and computation resources are allocated in order to minimize energy consumption.In this paper, a brief survey on U-MEC networks is presented. It includes the brief introduction regarding UAVs and MEC technology. The basic terminologies and architectures used in U-MEC networks are also defined. Moreover, mobile edge computation offloading working, different access schemes used during computation offloading technique are explained. Resources that are needed to be optimized in U-MEC systems are depicted with different optimization problem, and solution types. Furthermore, to guide future work in this area of research, future research directions are outlined. At the end, challenges and open issues in this domain are also summarized.

show abstract

“…Reinforcement Learning [17] can learn and optimize for UAV-assisted MEC without training data by interacting with MEC environment. Therefore, reinforcement learning [17] and Deep Reinforcement Learning (DRL) [18] methods are introduced resource allocation and UAV position optimization [19] to minimize energy consumption. DRL [20] employs deep neural networks to capture the complex states of UAV-assisted MEC and reinforcement learning to make decisions.…”

Section: Introductionmentioning

confidence: 99%

Task Offloading and Trajectory Control for UAV-Assisted Mobile Edge Computing Using Deep Reinforcement Learning

Zhang

Luo

et al. 2021

IEEE Access

View full text Add to dashboard Cite

Mobile Edge Computing (MEC) has been widely employed to support various Internet of Things (IoT) and mobile applications. By leveraging the advantages of easily deployed and flexibility of Unmanned Aerial Vehicle (UAV), one of MEC primary functions is employing UAVs equipped with MEC servers to provide computation supports for the offloaded tasks by mobile users in temporally hotspot areas or some emergent scenarios, such as sports game areas or destroyed by natural disaster areas. Despite the numerous advantages of UAV carried with a MEC server, it is restricted by its limited computation resources and sensitive energy consumption. Moreover, due to the complexity of UAV-assisted MEC system, its computational resource optimization and energy consumption optimization cannot be achieved well in traditional optimization methods. Furthermore, the computational cost of the MEC system optimization is often exponentially growing with the increase of the MEC servers and mobile users. Therefore, it is considerably challenging to control the UAV positions and schedule the task offloading ratio. In this paper, we proposed a novel Deep Reinforcement Learning (DRL) method to optimize UAV trajectory controlling and users' offloaded task ratio scheduling and improve the performance of the UAV-assisted MEC system. We maximized the system stability and minimized the energy consumption and computation latency of UAV-assisted the MEC system. The simulation results show that the proposed method outperforms existing work and has better scalability.INDEX TERMS Computation offloading, deep learning, energy saving, mobile edge computing, reinforcement learning, unmanned aerial vehicle.

show abstract

Deep Reinforcement Learning Based Dynamic Trajectory Control for UAV-Assisted Mobile Edge Computing

Cited by 158 publications

References 37 publications

Evolutionary Multi-Objective Reinforcement Learning Based Trajectory Control and Task Offloading in UAV-Assisted Mobile Edge Computing

Evolutionary Multi-Objective Reinforcement Learning Based Trajectory Control and Task Offloading in UAV-Assisted Mobile Edge Computing

Energy Efficient UAV-Enabled Mobile Edge Computing for IoT Devices: A Review

Task Offloading and Trajectory Control for UAV-Assisted Mobile Edge Computing Using Deep Reinforcement Learning

Contact Info

Product

Resources

About