A computation offloading method with distributed double deep Q‐network for connected vehicle platooning with vehicle‐to‐infrastructure communications

Abstract

Current connected vehicle applications, such as platooning require heavy‐load computing capability. Although mobile edge computing (MEC) servers connected to the roadside intelligence facility can assist such separable applications from vehicles, it is a challenge to coordinate the allocation of subtasks among vehicles and MEC servers on the premise of ensuring communication quality. Therefore, an offloading algorithm is proposed based on a double deep Q‐network to solve the placement of subtasks for vehicle to infrastructure and vehicle to vehicle cases. This algorithm considers the randomness of task generation and is model‐free. The MEC server can assist the vehicle in training the neural network and storing relevant state transitions. To improve the performance of the algorithm, the decaying greedy policy is incorporated for faster convergence. The simulation results showed that this algorithm performed well in reducing the dropped subtask rate, average time delay, and total energy consumption.

Deep Reinforcement Learning-Based Task Offloading and Load Balancing for Vehicular Edge Computing

“…Zheng et al [23] developed an asynchronous dominant participant-critic-based decision-making algorithm in a digital twin network framework aiming at fast convergence and reduced system cost. Shi et al [10] proposed an offloading algorithm based on a dual-depth Q network to solve the problem of offloading subtasks between vehicles and subtasks between vehicles and edge nodes with the aim of reducing the subtask packet loss rate, the average task delay, and the total energy consumption. Liu et al [24] accounted for dependencies between subtasks by modeling these dependencies with a directed acyclic graph and proposed a task-offloading algorithm based on a deep deterministic policy gradient (DDPG).…”

“…However, the following issues still need to be explored. First, due to the high communication and storage costs of MEC servers, particularly when a high number of vehicles gather within their communication coverage, unloading vehicle tasks to MEC servers may have the opposite effect [10]. In addition, the mobility of vehicles in a VEC network and the disparity in regional infrastructure deployments can lead to load imbalances between MEC servers.…”