Deep Reinforcement Learning-Based Spectrum Allocation Algorithm in Internet of Vehicles Discriminating Services

Abstract

With the rapid development of global automotive industry intelligence and networking, the Internet of Vehicles (IoV) service, as a key communication technology, has been faced with an increasing spectrum of resources shortage. In this paper, we consider a spectrum utilization problem, in which a number of co-existing cellular users (CUs) and prioritized device-to-device (D2D) users are equipped in a single antenna vehicle-mounted communication network. To ensure a business-aware spectrum access mechanism with delay granted in a complex dynamic environment, we consider optimizing a metric that maintains a trade off between maximizing the total capacity of vehicle to vehicle (V2V) and vehicle to infrastructure (V2I) links and minimizing the interference of high priority links. A low complexity priority-based spectrum allocation scheme based on the deep reinforcement learning method is developed to solve the proposed formulation. We trained our algorithm using the deep Q-learning network (DQN) over a set of public bandwidths. Simulation results show that the proposed scheme can allocate spectrum resources quickly and effectively in a high dynamic vehicle network environment. Concerning improved channel transmission rate, the V2V link rate in this scheme is 2.54 times that of the traditional random spectrum allocation scheme, and the V2I link rate is 13.5% higher than that of the traditional random spectrum allocation scheme. The average total interference received by priority links decreased by 14.2 dB compared to common links, realized service priority distinction and has good robustness to communication noise.

Unleashing the Power of IoT: A Comprehensive Review of IoT Applications and Future Prospects in Healthcare, Agriculture, Smart Homes, Smart Cities, and Industry 4.0

“…In particular, machine learning and artificial intelligence have played key roles in advancing this emerging field of computer science and will continue to do so in new and exciting ways [ 57 ]. It is critical for computer scientists to understand common machine learning and AI algorithms that are being used, as well as consider what research is currently being conducted to advance IoT technologies [ 58 , 59 , 60 , 61 , 62 , 63 , 64 , 65 , 66 , 67 , 68 , 69 , 70 , 71 , 72 , 73 ].…”

Unleashing the Power of IoT: A Comprehensive Review of IoT Applications, Advancements, and Future Prospects in Healthcare, Agriculture, Smart Homes, Smart Cities, and Industry 4.0

“…In particular, machine learning and artificial intelligence have played key roles in advancing this emerging field of computer science and will continue to do so in new and exciting ways [57]. It is critical for computer scientists to understand common machine learning and AI algorithms that are being used, as well as consider what research is currently being conducted to advance IoT technologies [58][59][60][61][62][63][64][65][66][67][68][69][70][71][72][73].…”

Shear Wave Velocity Estimation Based on Deep-Q Network

“…DRL has been widely used as an intelligent controller for different purposes, including robotics [14], electronic sports [15], automatic controlling [16][17][18][19][20][21][22][23], etc. Specifically, Gu et al demonstrated the effectiveness of DRL for controlling physical robots [14].…”

“…Guan et al designed a DRL-based spectrum allocation algorithm for the internet of vehicles discriminating services. It has been proven that the designed method allocates spectrum resources quickly and efficiently in a highly dynamic environment [20]. Zhao et al utilized DQN for controlling the autonomous walking of an underground load-haul-dump machine and demonstrated the effectiveness of the proposed DQN-based method through experimental verification [21].…”