Deep-Reinforcement-Learning-Based Mode Selection and Resource Allocation for Cellular V2X Communications

Abstract

Cellular vehicle-to-everything (V2X) communication is crucial to support future diverse vehicular applications. However, for safety-critical applications, unstable vehicle-to-vehicle (V2V) links and high signalling overhead of centralized resource allocation approaches become bottlenecks. In this paper, we investigate a joint optimization problem of transmission mode selection and resource allocation for cellular V2X communications. In particular, the problem is formulated as a Markov decision process, and a deep reinforcement learning (DRL) based decentralized algorithm is proposed to maximize the sum capacity of vehicle-to-infrastructure users while meeting the latency and reliability requirements of V2V pairs. Moreover, considering training limitation of local DRL models, a two-timescale federated DRL algorithm is developed to help obtain robust model. Wherein, the graph theory based vehicle clustering algorithm is executed on a large timescale and in turn the federated learning algorithm is conducted on a small timescale. Simulation results show that the proposed DRL-based algorithm outperforms other decentralized baselines, and validate the superiority of the twotimescale federated DRL algorithm for newly activated V2V pairs.

Federated Learning for Internet of Things: A Comprehensive Survey

“…Compared to centralized learning approaches, FL offers more benefits for intelligent vehicular services such as better efficiency in resource usage, lower power consumption with a similar learning accuracy. Another resource allocation scheme for vehicle-toeverything (V2X) communication is also considered in [135]. In this setting, FL is combined with DRL [136] to build a federated intelligent resource allocation strategy, in order to maximize the sum capacity of vehicular users with respect to latency and reliability conditions.…”

Deep Reinforcement Learning for Radio Resource Allocation and Management in Next Generation Heterogeneous Wireless Networks: A Survey

“…In another interesting work in [160], the authors study the problem of joint optimization of transmission mode selection and resource allocation for CV2X. They propose singleagent settings in which DQN and federated learning (FL) models are integrated to improve the model's robustness.…”

Joint Channel Allocation and Routing for ZigBee/Wi-Fi Coexistent Networks

“…X. Zhang et al [41] apply deep reinforcement learning to V2V communication and Vehicle-to-Infrastructure (V2I) communication to find optimal allocation of communication resource and transmission power to vehicles in addition to selection of V2V communication or V2I communication.…”

“…These works [37]- [41] incorporate power control into their proposed methods, and assume D2D communication underlaying cellular networks. On the other hand, this paper focuses on not power control but channel allocation and routing.…”