A Vehicle-Assisted Computation Offloading Algorithm Based on Proximal Policy Optimization in Vehicle Edge Networks

Abstract

With the continuous development of the Internet of Vehicles(IoV), Vehicle Edge Computing(VEC) has become a key technology for computational resource scheduling, but more and more smart devices are connected to the internet, which makes it difficult for traditional Vehicle Edge Networks(VEN) to deal with tasks in time. In this paper, in order to cope with the challenges of the large number of devices accessing the internet, we propose a vehicle-assisted computation offloading algorithm based on proximal policy optimization(VCOPPO) for User Equipment(UE) tasks, and it combines dynamic parked vehicles incentives mechanism and computational resource allocation strategy by using road vehicles and parked vehicles as edge servers. Firstly, a non-convex optimization problem combining VEN utility and task processing delay is formulated, subject to the constraints of the residual energy and the transmission rate of the task. Secondly, the proposed VCOPPO is used to solve the formulated non-convex optimization problem, and we use stochastic policy to obtain the optimal computation offloading decisions and resource allocation schemes. Finally, the experimental results have shown that the proposed VCOPPO has an excellent performance in network reward and task processing delay respectively, and it can effectively schedule and allocate computational resources. Compared with using Dueling Deep Q Network(Dueling DQN), Deep Q Network(DQN) and Q-learning methods, the proposed VCOPPO improves the network reward by 31%, 18% and 91%, reduces the delay in task processing by 78%, 63% and 74%, respectively.

Federated Reinforcement Learning in IoT: Applications, Opportunities and Open Challenges

“…Deep Q-learning [44] and double deep Q-learning (DDQN) [45] have supported transmission scheduling in IoT [46,47]. Proximal policy optimization (PPO) [48] and advantage actor-critic (A2C) [49,50] were adopted for computation offloading [51,52], while the deep deterministic policy gradient (DDPG) [53] can be adopted for intrusion detection [54] in green IoT. Finally, the asynchronous advantage actor-critic (A3C) [55] has been adopted in service placement [56].…”

Intelligent data offloading learning model for autonomous vehicular networks

“…A vehicle‐assisted computation offloading algorithm 24 for user equipment (UE) tasks based on proximal policy optimization (VCOPPO). A combination resource dissemination along with reward with the help of edge servers that are vehicles that has been parked or at the roadside.…”