Multi-Objective Optimization for UAV-Assisted Wireless Powered IoT Networks Based on Extended DDPG Algorithm

Abstract

This paper studies an unmanned aerial vehicle (UAV)-assisted wireless powered IoT network, where a rotarywing UAV adopts fly-hover-communicate protocol to successively visit IoT devices in demand. During the hovering periods, the UAV works on full-duplex mode to simultaneously collect data from the target device and charge other devices within its coverage. Practical propulsion power consumption model and non-linear energy harvesting model are taken into account. We formulate a multi-objective optimization problem to jointly optimize three objectives: maximization of sum data rate, maximization of total harvested energy and minimization of UAV's energy consumption over a particular mission period. These three objectives are in conflict with each other partly and weight parameters are given to describe associated importance. Since IoT devices keep gathering information from the physical surrounding environment and their requirements to upload data change dynamically, online path planning of the UAV is required. In this paper, we apply deep reinforcement learning algorithm to achieve online decision. An extended deep deterministic policy gradient (DDPG) algorithm is proposed to learn control policies of UAV over multiple objectives. While training, the agent learns to produce optimal policies under given weights conditions on the basis of achieving timely data collection according to the requirement priority and avoiding devices' data overflow. The verification results show that the proposed MODDPG (multiobjective DDPG) algorithm achieves joint optimization of three objectives and optimal policies can be adjusted according to weight parameters among optimization objectives.

Evolutionary Multi-Objective Reinforcement Learning Based Trajectory Control and Task Offloading in UAV-Assisted Mobile Edge Computing

“…A singlepolicy MORL aims to optimize one policy for a given preference. For example, the authors in [31] extended a single-objective DDPG to a single-policy MORL to optimize the data rate, total harvested energy, and UAV's energy consumption. However, a single-policy MORL cannot output multiple optimal policies after a run, each of which optimizes a certain preference.…”

Multi-Objective UAV Positioning Mechanism for Sustainable Wireless Connectivity in Environments with Forbidden Flying Zones

“…In addition, recent research looks on multiobjective optimization of UAV-assisted communication [48,49]. Over the course of a flight, a multiobjective optimization problem is constructed to jointly optimize three objectives [49]: (1) maximization of cumulative data rate, (2) maximization of total gathered energy, and (3) reduction of UAV energy consumption.…”

“…In addition, recent research looks on multiobjective optimization of UAV-assisted communication [48,49]. Over the course of a flight, a multiobjective optimization problem is constructed to jointly optimize three objectives [49]: (1) maximization of cumulative data rate, (2) maximization of total gathered energy, and (3) reduction of UAV energy consumption. Because these goals are incompatible, the authors suggested an enhanced deep deterministic policy gradient (DDPG) technique for learning UAV control policies with multiple goals.…”

Rate Maximization in a UAV Based Full-Duplex Multi-User Communication Network Using Multi-Objective Optimization

“…In recent works, effectiveness of FD in a UAV network was shown in [19], where the authors considered maximizing the DL sum rate while maintaining the minimum quality-ofservice (QoS) for the UL users, by optimizing the resources and location of the UAV. In [20], FD was considered in a UAV-assisted wireless powered Internet-of-Things (IoT) network, where the FD UAV collects data from the target IoT device and simultaneously charges remaining devices utilizing fly-hover-communicate protocol. Moreover, a FD UAV relay network was considered in [21], to jointly optimize the HD UL and DL users scheduling, and the UAV trajectory.…”

“…The optimization problem is non-convex and non-tractable. MOOP was also considered in [20], where a deep learning approach is used to solve the conflicting system objectives. Contrary, we first adopt weighted Tchebycheff method to convert MOOP to a single-objective-optimization-problem (SOOP).…”