Data Freshness and Energy-Efficient UAV Navigation Optimization: A Deep Reinforcement Learning Approach

Recently, using unmanned aerial vehicles (UAVs) to collect information from distributed sensors has become one of the hotspots in the Internet of Things (IoT) research. However, previous studies on the UAV-assisted data acquisition systems focused mainly on shortening the acquisition time, reducing the energy consumption, and increasing the amount of collected data, but it lacked the optimization of data freshness. Moreover, we hope that UAVs can perform long-term data collection tasks in dynamic scenarios within a constantly changing age of information (AoI) and within their own power levels. Therefore, we aim to maximize the quality of service (QoS) based on the freshness of data, while considering the endurance of the UAVs. Since our scenario is not an inertial order decision process with uniform time slots, we first transform the optimization problem into a semi-Markov decision process (SMDP) through modeling, and then we propose a hierarchical deep Q-network (DQN)-based path-planning algorithm to learn the optimal strategy. The simulation results show that the algorithm is better than the benchmark algorithm, and the tradeoff between the system QoS and the safe power state can be achieved by adjusting the parameter βe.

show abstract

“…according to [35], we define the function u(∆) = a w∆ , where 0 < a < 1 is a constant, and w is the time-sensitive weight.…”

Section: Freshness Of Collected Datamentioning

confidence: 99%

The UAV Trajectory Optimization for Data Collection from Time-Constrained IoT Devices: A Hierarchical Deep Q-Network Approach

Qin

Zhang

et al. 2022

Applied Sciences

View full text Add to dashboard Cite

show abstract

“…[119] utilized massive MIMO communication to guide UAVs on their optimal path using MDP-based DRL. Similarly, in[124], Abedin et al proposed an MDP-based DRL approach for UAV-BS navigation considering data freshness and energy-efficient. The authors derived the UAV-BS navigation problem as an NP-hard problem using DRL with experience reply memory.…”

mentioning

confidence: 99%

Artificial Intelligence Approaches for UAV Navigation: Recent Advances and Future Challenges

Rezwan

Choi

2022

IEEE Access

View full text Add to dashboard Cite

Unmanned aerial vehicles (UAVs) applications have increased in popularity in recent years because of their ability to incorporate a wide variety of sensors while retaining cheap operating costs, easy deployment, and excellent mobility. However, controlling UAVs remotely in complex environments limits the capability of the UAVs and decreases the efficiency of the whole system. Therefore, many researchers are working on autonomous UAV navigation where UAVs can move and perform the assigned tasks based on their surroundings. With recent technological advancements, the application of artificial intelligence (AI) has proliferated. Autonomous UAV navigation is an example of an application in which AI plays a critical role in providing fundamental human control characteristics. Thus, many researchers have adopted different AI approaches to make autonomous UAV navigation more efficient. This paper comprehensively surveys and categorizes several AI approaches for autonomous UAV navigation implicated by several researchers. Different AI approaches comprise mathematical-based optimization and model-based learning approaches. The fundamentals, working principles, and main features of the different optimization-based and learningbased approaches are discussed in this paper. In addition, the characteristics, types, navigation models, and applications of UAVs are highlighted to make AI implementation understandable. Finally, the open research directions are discussed to provide researchers with clear and direct insights for further research.

show abstract

“…On the other hand, AoI with energy-limited UAV trajectory planning is studied in the literature [19][20][21][22][23][24]. For instance, Jia et al [19] employed the concept of age to study a UAV path planning and data acquisition problem by jointly considering the data acquisition mode selection, energy consumption at each node and age evolution of collected information.…”

mentioning

confidence: 99%

Age‐optimal path planning for finite‐battery UAV‐assisted data dissemination in IoT networks

Changizi

Emadi

2021

IET Communications

View full text Add to dashboard Cite

Unmanned aerial vehicles have been widely used to assist wireless sensor networks due to ever-increasing demands for Internet-of-things applications. To support timely delivery of information characterised by a recently introduced metric, termed as the age of information, this paper explores freshness of data in an unmanned aerial vehicle assisted wireless sensor network. Specifically, the authors consider a limited-energy unmanned aerial vehicle moving towards the Internet-of-things devices to disseminate data packets provided by a data centre. Since the unmanned aerial vehicle cannot visit all the nodes in each flight turn due to its finite-sized battery, the best sequence of nodes, from an age of information perspective, should be selected at the beginning of each flight turn. Thus, an unmanned aerial vehicle trajectory planning for data dissemination is proposed taking into account both maximal use of energy and freshness of data. To minimise the weighted sum age of information metric, by utilising the well-known knapsack and travelling salesman problems, the authors propose an algorithm to efficiently select devices and the corresponding visiting order in each flight turn. Finally, to highlight performance of the proposed algorithm, and to investigate the effect of limited-energy unmanned aerial vehicles, the number of nodes and flight turns, and simulation results are also provided and compared with other benchmark algorithms. INTRODUCTIONRecently, unmanned aerial vehicles (UAVs) have attracted a lot of research attention in many Internet-of-things (IoT) services such as real-time monitoring of traffic [1], disaster management [2] and crowd surveillance [3]. Due to the high mobility and lineof-sight (LoS) communication opportunities, UAVs can be used as a flying base station (BS) for data collection or dissemination, specifically enhancing connectivity when there is no good enough reliable communications link between IoT devices and a BS [4]. However, the deployment of UAVs as aerial BSs leads to several challenges such as optimal trajectory planning, placement and energy consumption [5]. These challenges have been extensively studied in the literature from various perspectives: minimising delay and flight time or maximising network coverage and throughput. For instance, Mozaffari et al. [6] studied optimal deployment of UAVs to maximise network coverage. In [7], the authors min-This is an open access article under the terms of the Creative Commons Attribution License, which permits use, distribution and reproduction in any medium, provided the original work is properly cited.

show abstract

Data Freshness and Energy-Efficient UAV Navigation Optimization: A Deep Reinforcement Learning Approach

Cited by 79 publications

References 36 publications

The UAV Trajectory Optimization for Data Collection from Time-Constrained IoT Devices: A Hierarchical Deep Q-Network Approach

The UAV Trajectory Optimization for Data Collection from Time-Constrained IoT Devices: A Hierarchical Deep Q-Network Approach

Artificial Intelligence Approaches for UAV Navigation: Recent Advances and Future Challenges

Age‐optimal path planning for finite‐battery UAV‐assisted data dissemination in IoT networks

Contact Info

Product

Resources

About