Impact of Quantized Side Information on Subchannel Scheduling for Cellular V2X

Abanto-Leon, Luis F.; Koppelaar, Arie; Math, Chetan Belagal; Groot, Sonia Heemstra de

doi:10.1109/vtcspring.2018.8417882

Cited by 6 publications

(1 citation statement)

References 3 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, the optimal policies are normally time-dependent in a finite-horizon setting, while DDPG is designed to solve the infinite-horizon or indefinite-horizon problems, where the learned policy is the same for every time step [22]. In order to deal with this problem, we set the target values of DDPG in the last control interval K −1 to be derived by (28) in the way as for the other control intervals, i.e., the sum of the immediate reward and the discounted target Q value of the next state instead of only the immediate reward…”

Section: Mtcc-pc Algorithmmentioning

confidence: 99%

Jointly Learning V2X Communication and Platoon Control with Deep Reinforcement Learning

Liu,

Lei,

Liu

et al. 2023

2023 IEEE 34th Annual International Symposium on Personal, Indoor and Mobile Radio Communications (PIMRC)

View full text Add to dashboard Cite

An intelligent decision-making system enabled by Vehicle-to-Everything (V2X) communications is essential to achieve safe and efficient autonomous driving (AD), where two types of decisions have to be made at different timescales, i.e., vehicle control and radio resource allocation (RRA) decisions. The interplay between RRA and vehicle control necessitates their collaborative design. In this two-part paper (Part I and Part II), taking platoon control (PC) as an example use case, we propose a joint optimization framework of multi-timescale control and communications (MTCC) based on Deep Reinforcement Learning (DRL). In this paper (Part I), we first decompose the problem into a communication-aware DRL-based PC sub-problem and a control-aware DRL-based RRA sub-problem. Then, we focus on the PC sub-problem assuming an RRA policy is given, and propose the MTCC-PC algorithm to learn an efficient PC policy. To improve the PC performance under random observation delay, the PC state space is augmented with the observation delay and PC action history. Moreover, the reward function with respect to the augmented state is defined to construct an augmented state Markov Decision Process (MDP). It is proved that the optimal policy for the augmented state MDP is optimal for the original PC problem with observation delay. Different from most existing works on communication-aware control, the MTCC-PC algorithm is trained in a delayed environment generated by the fine-grained embedded simulation of C-V2X communications rather than by a simple stochastic delay model. Finally, experiments are performed to compare the performance of MTCC-PC with those of the baseline DRL algorithms.

show abstract

Section: Mtcc-pc Algorithmmentioning

confidence: 99%