With the development of 6G, the rapidly increasing number of smart devices deployed in the Industrial Internet of Things (IIoT) environment has been witnessed. The radio environment is showing a trend of complexity, and spectrum conflicts are becoming increasingly acute. User equipment (UE) can accurately sense and utilize spectrum resources through radio map (RM). However, the construction and dissemination of RM incur a heavy computational burden and large dissemination delay, which limit the real-time sensing of spatial spectrum situations. In this paper, we propose an RM construction and dissemination method based on deep reinforcement learning (DRL) in the context of mobile edge computing (MEC) networks. We formulate the dissemination modes selection and resource allocation problems during RM construction and dissemination as a mixed-integer nonlinear programming problem. Then, we propose an actor-critic-based joint offloading and resource allocation (ACJORA) algorithm for intelligent scheduling of computational offloading and resource allocation. We design a novel weighted loss function for the actor network, which combines the discrete actions for offloading decisions and the continuous actions for resource allocation. And the simulation results show that the proposed algorithm can reduce the cost of dissemination by optimizing the offloading strategies and resources, which is more applicable for real-time RM applications in MEC networks.

With the emergence of a large number of smart devices, the radio environment in which unmanned aerial vehicles (UAVs) take tasks is becoming more and more complex, which puts forward higher requirements for UAVs’ situational awareness and autonomous obstacle avoidance capabilities. To tackle this issue, we propose a three-dimension (3D) UAV path planning method under communication connectivity constraints guided by radio environment maps (REMs), which are distributed by ground edge servers in the form of compressed global REMs and detailed local REMs. An interfered fluid dynamic system (IFDS) model is deployed on UAVs to allow them to avoid obstacles and plan paths. We propose a twin-delayed deep deterministic policy gradient- (TD3-) based deep reinforcement learning (DRL) method to optimize the reaction coefficients of UAVs to avoid obstacles and improve the signal to interference plus noise ratio (SINR). The simulation results show that the proposed algorithm can effectively avoid static obstacles and dynamic interference under communication connectivity constraints, significantly improve the communication stability with a higher receive signal SINR and reduce the cost of UAV performing tasks with the shortest path.

This paper investigates a deep reinforcement learning algorithm based on dueling deep recurrent Q -network (Dueling DRQN) for dynamic multichannel access in heterogeneous wireless networks. Specifically, we consider the scenario that multiple heterogeneous users with different MAC protocols share multiple independent channels. The goal of the intelligent node is to learn a channel access strategy that achieves high throughput by making full use of the underutilized channels. Two key challenges for the intelligent node are (i) there is no prior knowledge of spectrum environment or the other nodes’ behaviors; (ii) the spectrum environment is partially observable, and the spectrum states have complex temporal dynamics. In order to overcome the aforementioned challenges, we first embed the long short-term memory layer (LSTM) into the deep Q -network (DQN) to aggregate historical observations and capture the underlying temporal feature in the heterogeneous networks. And second, we employ the dueling architecture to overcome the observability problem of dynamic environment in neural networks. Simulation results show that our approach can learn the optimal access policy in various heterogeneous networks and outperforms the state-of-the-art policies.