Efficient Drone Mobility Support Using Reinforcement Learning

Abstract

Flying drones can be used in a wide range of applications and services from surveillance to package delivery. To ensure robust control and safety of drone operations, cellular networks need to provide reliable wireless connectivity to drone user equipments (UEs). To date, existing mobile networks have been primarily designed and optimized for serving ground UEs, thus making the mobility support in the sky challenging. In this paper, a novel handover (HO) mechanism is developed for a cellular-connected drone system to ensure robust wireless connectivity and mobility support for drone-UEs. By leveraging tools from reinforcement learning, HO decisions are dynamically optimized using a Q-learning algorithm to provide an efficient mobility support in the sky. The results show that the proposed approach can significantly reduce (e.g., by 80%) the number of HOs, while maintaining connectivity, compared to the baseline HO scheme in which the drone always connects to the strongest cell.

A deep learning approach to efficient drone mobility support

“…In [24], HO measurements were reported for an aerial drone connected to an LTE network in a suburban environment. In our recent work [15], a HO mechanism based on Q-learning was proposed for a cellular-connected drone network. It was shown that a significant reduction in the number of HOs is attained while maintaining reliable connectivity.…”

“…It was shown that a significant reduction in the number of HOs is attained while maintaining reliable connectivity. Despite the encouraging results in [15], the tabular Q-learning framework adopted in [15] may have some disadvantages, such as predefined waypoints and high storage requirements when state space is large.…”

“…The main contribution of this paper is a novel deep Q-network (DQN) based optimization mechanism for a cellular-connected drone system to ensure robust connectivity for drone UEs by overcoming the disadvantages in the scheme proposed in [15]. With the use of deep RL tools, HO decisions are dynamically optimized using a deep neural network to provide an efficient mobility support for drones.…”

“…This, in turn, can trigger frequent HOs leading to undesirable outcomes such as radio link failures, ping-pong HOs, and large signaling overheads. This motivates the need for an efficient HO mechanism that can provide a robust drone mobility support in the sky [15].…”

“…We note that the RL-based HO algorithm merely expects that the drone trajectory is known beforehand. We generate the trajectory randomly as explained in [15].…”