Reinforcement Learning-Based Resource Allocation for Multiple Vehicles with Communication-Assisted Sensing Mechanism

Abstract

Autonomous vehicles (AVs) can be equipped with Integrated sensing and communications (ISAC) devices to realize sensing and communication functions simultaneously. Time-division ISAC (TD-ISAC) is advantageous due to its ease of implementation, efficient deployment and integration into any system. TD-ISAC greatly enhances spectrum efficiency and equipment utilization and reduces system energy consumption. In this paper, we propose a communication-assisted sensing mechanism based on TD-ISAC to support multi-vehicle collaborative sensing. However, there are some challenges in applying TD-ISAC to AVs. First, AVs should allocate resources for sensing and communication in a dynamically changing environment. Second, the limited spectrum resources bring the problem of mutual interference of multi-vehicle signals. To address these issues, we construct a multi-vehicle signal interference model, formulate an optimization problem based on the partially observable Markov decision process (POMDP) framework and design a decentralized dynamic allocation scheme for multi-vehicle time–frequency resources based on a deep reinforcement learning (DRL) algorithm. Simulation results show that the proposed scheme performs better in miss detection probability and average system interference power compared to the DRQN algorithm without the communication-assisted sensing mechanism and the random algorithm without reinforcement learning. We can conclude that the proposed scheme can effectively allocate the resources of the TD-ISAC system and reduce interference between multiple vehicles.