Joint Trajectory and Power Optimization for Energy Efficient UAV Communication Using Deep Reinforcement Learning

Cui, Yuling; Deng, Danhao; Wang, Chaowei; Wang, Weidong

doi:10.1109/infocomwkshps51825.2021.9484490

Cited by 10 publications

(3 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In [101] a deep deterministic policy gradient (DDPG)-based algorithm was used to optimize the overall uplink throughput and energy consumption where the state constituted an HAP equipped with MEC server multiple UAVs. Cui et al in [102] also used deep deterministic policy gradient (DDPG) algorithm for UAV trajectory design and power allocation to maximize the downlink throughput service time considering UAVs as aerial base stations.…”

Section: ) Enhanced Qos/qoementioning

confidence: 99%

Reinforcement Learning in the Sky: A Survey on Enabling Intelligence in NTN-Based Communications

et al. 2023

View full text Add to dashboard Cite

Non terrestrial networks (NTN) involving 'in the sky' objects such as low-earth orbit satellites, high altitude platform systems (HAPs) and Unmanned Aerial Vehicles (UAVs) are expected to be integral components of next generation cellular systems. With the deployment of 5G services and beyond, NTNs are leveraged to assist as aerial base stations in providing ubiquitous network connectivity and service to ground users or be deployed as aerial users connected to the cellular network. NTN-aided wireless communication offers multiple benefits such as mobility, flexibility, resistance to ground physical attacks and wide coverage. However, due to their limited resources and the current design of terrestrial cellular systems that do not account for aerial users, and other restrictions such as service requirements, limited available power and storage resources on high-throughput satellites, resource allocation, location of the high altitude platform base station and the flight trajectory of the UAVs need to be intelligently controlled to satisfy various objectives both from an aerial base station and overall network perspectives. To achieve this, many works have explored Reinforcement Learning (RL) techniques to allow aerial platforms in non-terrestrial networks to learn from past observations and achieve some optimal control policy. In this paper and differently from prior surveys, we contribute a comprehensive review of the control objectives required by non-terrestrial platforms that have been solved using RL formulations. We provide an up-to-date overview of the latest applications of RL techniques for different NTN-aided wireless communication aspects. The survey focuses on Markov Decision Process (MDP) formulations in terms of states, actions, and rewards. We synthesize a taxonomy from the surveyed literature and provide a comprehensive representation of the current usages of RL in NTN-aided wireless communications. A qualitative analysis of the level of realism achieved in the works presented in the literature is provided based on several factors that pertain to the simulation environment, station deployment setting, wireless channel assumption, and energy considerations. We also curate a list of challenges that remain to be considered by the research community in order to achieve more efficient deployments and close the simulation-to-reality gap.

show abstract

Section: ) Enhanced Qos/qoementioning

confidence: 99%

Reinforcement Learning in the Sky: A Survey on Enabling Intelligence in NTN-Based Communications

et al. 2023

View full text Add to dashboard Cite

show abstract

“…A combination of support vector regression (SVR) and k-means algorithm was first proposed to determine the optimal multicast grouping, after which a centroid adjustable traveling salesman-based algorithm was proposed to determine the optimal trajectory to maximize the energy-saving performance of the UAV. A joint trajectory and power optimization scheme based on DRL was proposed in [236] to maximize both the EE and throughput in a UAV-based communication system. The proposed scheme employs DDPG to solve the optimization problem in order to achieve the desired objective.…”

Section: Machine Learning Approachesmentioning

confidence: 99%

A Survey on Energy Optimization Techniques in UAV-Based Cellular Networks: From Conventional to Machine Learning Approaches

Abubakar

Ahmad

Omeke

et al. 2023

Drones

View full text Add to dashboard Cite

Wireless communication networks have been witnessing unprecedented demand due to the increasing number of connected devices and emerging bandwidth-hungry applications. Although there are many competent technologies for capacity enhancement purposes, such as millimeter wave communications and network densification, there is still room and need for further capacity enhancement in wireless communication networks, especially for the cases of unusual people gatherings, such as sport competitions, musical concerts, etc. Unmanned aerial vehicles (UAVs) have been identified as one of the promising options to enhance capacity due to their easy implementation, pop-up fashion operation, and cost-effective nature. The main idea is to deploy base stations on UAVs and operate them as flying base stations, thereby bringing additional capacity where it is needed. However, UAVs mostly have limited energy storage, hence, their energy consumption must be optimized to increase flight time. In this survey, we investigate different energy optimization techniques with a top-level classification in terms of the optimization algorithm employed—conventional and machine learning (ML). Such classification helps understand the state-of-the-art and the current trend in terms of methodology. In this regard, various optimization techniques are identified from the related literature, and they are presented under the above-mentioned classes of employed optimization methods. In addition, for the purpose of completeness, we include a brief tutorial on the optimization methods and power supply and charging mechanisms of UAVs. Moreover, novel concepts, such as reflective intelligent surfaces and landing spot optimization, are also covered to capture the latest trends in the literature.

show abstract

“…In [37,38], the authors emphasize the importance of UAV trajectory design, which affects not only the communication and link budget between UAVs and ground users, but also UAVs' energy consumption. They also considered a deep-reinforcement algorithm for trajectory design and power allocation optimization.…”

Section: Related Workmentioning

confidence: 99%

Modifying Hata-Davidson Propagation Model for Remote Sensing in Complex Environments Using a Multifactional Drone

Almalki

Soufiene

2022

Sensors

View full text Add to dashboard Cite

The coupling of drones and IoT is a major topics in academia and industry since it significantly contributes towards making human life safer and smarter. Using drones is seen as a robust approach for mobile remote sensing operations, such as search-and-rescue missions, due to their speed and efficiency, which could seriously affect victims’ chances of survival. This paper aims to modify the Hata-Davidson empirical propagation model based on RF drone measurement to conduct searches for missing persons in complex environments with rugged areas after manmade or natural disasters. A drone was coupled with a thermal FLIR lepton camera, a microcontroller, GPS, and weather station sensors. The proposed modified model utilized the least squares tuning algorithm to fit the data measured from the drone communication system. This enhanced the RF connectivity between the drone and the local authority, as well as leading to increased coverage footprint and, thus, the performance of wider search-and-rescue operations in a timely fashion using strip search patterns. The development of the proposed model considered both software simulation and hardware implementations. Since empirical propagation models are the most adjustable models, this study concludes with a comparison between the modified Hata-Davidson algorithm against other well-known modified empirical models for validation using root mean square error (RMSE). The experimental results show that the modified Hata-Davidson model outperforms the other empirical models, which in turn helps to identify missing persons and their locations using thermal imaging and a GPS sensor.

show abstract

Joint Trajectory and Power Optimization for Energy Efficient UAV Communication Using Deep Reinforcement Learning

Cited by 10 publications

References 22 publications

Reinforcement Learning in the Sky: A Survey on Enabling Intelligence in NTN-Based Communications

Reinforcement Learning in the Sky: A Survey on Enabling Intelligence in NTN-Based Communications

A Survey on Energy Optimization Techniques in UAV-Based Cellular Networks: From Conventional to Machine Learning Approaches

Modifying Hata-Davidson Propagation Model for Remote Sensing in Complex Environments Using a Multifactional Drone

Contact Info

Product

Resources

About