Maximization of instantaneous transmission rate in unmanned aerial vehicles‐supported self‐organized device‐to‐device network

Abstract

Summary Unmanned aerial vehicle (UAV)‐aided aerial base stations have emerged as a promising technique to provide rapid on‐demand wireless coverage for ground communicating devices in a geographical area. However, existing works on UAV‐enabled wireless communication systems overlook optimal deployment of UAVs under quality of service (QoS)‐aware device‐to‐device (D2D) communication. Therefore, this work proposes a UAV‐supported self‐organized device‐to‐device (USSD2D) network that employs multiple UAVs as relays for reliable D2D data transmission. The aim is to maximize the total instantaneous transmission rate of the USSD2D network by jointly optimizing devices' association with UAV, UAVs' channel selection, and their deployed location under signal‐to‐interference‐noise ratio (SINR) threshold. As this joint optimization problem is nonconvex and combinatorial, the formulated problem is transformed into a Markov decision process (MDP) that effectively splits up it into three individual optimization subproblems: devices association, UAVs' channel selection indicator, and UAVs' location at each instance. Finally, a reinforcement learning (RL) based on a low‐complexity iterative state–action–reward–state–action (SARSA) algorithm is developed to update UAVs' policy to solve this formulated problem. UAVs adopt the system parameters according to the current state and corresponding action to maximize the generated long‐term discounted reward under the current policy without prior knowledge about the environment. Numerical results validate the proposed approaches and provide various insights on optimal UAV deployment. This investigation demonstrates that the total instantaneous transmission rate of the USSD2D network can be improved by 75.25%, 51.31%, and 13.96% with respect to RS‐FORD, ES‐FIRD, and AOIV schemes, respectively.

Optimal Unmanned Aerial Vehicle Control and Designs for Load Balancing in Intelligent Wireless Communication Systems

“…Later, we compare the obtained results corresponding to the proposed SARSA algorithm with the existing works [34], such as random selection with fixed optimal relay deployment (RS-FORD), an exhaustive search for relay assignment and channel allocation with fixed initial relay deployment (ES-FIRD), and alternative optimization for the individual variable (AOIV). Here, we consider that direct D2D pair and UAV-assisted D2D pair devices are uniformly distributed in a 4 kmÂ4 km square area where the primary simulation parameters are adopted from [38].…”