Unmanned aerial vehicle (UAV) networks are playing an important role in various areas due to their agility and versatility, which have attracted significant attention from both the academia and industry in recent years. As an integration of the embedded systems with communication devices, computation capabilities and control modules, the UAV network could build a closed loop from data perceiving, information exchanging, decision making to the final execution, which tightly integrates the cyber processes into the physical devices. Therefore, the UAV network could be considered as a cyber physical system (CPS). Revealing the coupling effects among the three interacted components in this CPS system, i.e., communication, computation and control, is envisioned as the key to properly utilize all the available resources and hence improve the performance of the UAV networks. In this paper, we present a comprehensive survey on the UAV networks from a CPS perspective. Firstly, we respectively research the basics and advances with respect to the three CPS components in the UAV networks. Then we look inside to investigate how these components contribute to the system performance by classifying the UAV networks into three hierarchies, i.e., the cell level, the system level, and the system of system level. Further, the coupling effects among these CPS components are explicitly illustrated, which could be enlightening to deal with the challenges in each individual aspect. New research directions and open issues are discussed at the end of this survey. With this intensive literature review, we try to provide a novel insight into the state-of-the-art in the UAV networks.

No abstract

Making effective use of scarce spectrum resources, along with efficient computational performance, is one of the key challenges for future wireless networks. To tackle this issue, in this paper, we focus on the intelligent dynamic spectrum allocation (DSA) in a mobile edge computing (MEC) enabled cognitive network. And our objective is to optimize the spectrum utilization and load balance among idle channels. Since users can only acquire part of environment information in a decentralized way, we model such a problem as decentralized partially observed Markov decision process (Dec-POMDP) and design the corresponding evaluating metric to encourage users sense and access spectrum properly. Then, we propose a QMIX-based DSA method with centralized training decentralized execution (CTDE) structure to tackle it. In the training phase, the users offload the computational tasks to the MEC server to obtain the optimal distributed DSA strategies, through which the users select the optimal channel locally in the execution phase. Simulation results show that, using the proposed algorithm, users can independently capture spectrum holes, and hence improve the spectrum utilization while balancing the load on available channels.

No abstract

This paper investigates a deep reinforcement learning algorithm based on dueling deep recurrent Q -network (Dueling DRQN) for dynamic multichannel access in heterogeneous wireless networks. Specifically, we consider the scenario that multiple heterogeneous users with different MAC protocols share multiple independent channels. The goal of the intelligent node is to learn a channel access strategy that achieves high throughput by making full use of the underutilized channels. Two key challenges for the intelligent node are (i) there is no prior knowledge of spectrum environment or the other nodes’ behaviors; (ii) the spectrum environment is partially observable, and the spectrum states have complex temporal dynamics. In order to overcome the aforementioned challenges, we first embed the long short-term memory layer (LSTM) into the deep Q -network (DQN) to aggregate historical observations and capture the underlying temporal feature in the heterogeneous networks. And second, we employ the dueling architecture to overcome the observability problem of dynamic environment in neural networks. Simulation results show that our approach can learn the optimal access policy in various heterogeneous networks and outperforms the state-of-the-art policies.