Deep Reinforcement Learning for Collaborative Computation Offloading on Internet of Vehicles

Abstract

With the increase of Internet of vehicles (IoVs) traffic, the contradiction between a large number of computing tasks and limited computing resources has become increasingly prominent. Although many existing studies have been proposed to solve this problem, their main consideration is to achieve different optimization goals in the case of edge offloading in static scenarios. Since realistic scenarios are complicated and generally time-varying, these studies in static scenes are imperfect. In this paper, we consider a collaborative computation offloading in a time-varying edge-cloud network, and we formulate an optimization problem with considering both delay constraints and resource constraints, aiming to minimize the long-term system cost. Since the set of feasible solutions to the problem is nonconvex, and the complexity of the problem is very large, we propose a Q-learning-based approach to solve the optimization problem. In addition, due to the dimensional catastrophes, we further propose a DQN-based approach to solve the optimization problem. Finally, by comparing our two proposed algorithms with typical algorithms, the simulation results show that our proposed approaches achieve better performance. Under the same conditions, by comparing our two proposed algorithms with typical algorithms, the simulation results show that our proposed Q-learning-based method reduces the system cost by about 49% and 42% compared to typical algorithms. And in the same case, compared with the classical two schemes, our proposed DQN-based scheme reduces the system cost by 62% and 57%.

ASXC$^{2}$ Approach: A Service-X Cost Optimization Strategy Based on Edge Orchestration for IIoT

“…An alternate offloading technique is designed to trade off execution delay and energy usage. A collaborative service offloading model called greedy offloading and resource allocation (GORA) method is proposed in [19], leveraging Qlearning theory to optimize service cost while considering delay and resource constraints. Traditional solutions tend to be nonconvex and result in high complexity.…”

“…The count of communication request and response events for each service execution determines the communication overhead of the server, which is represented by ϱ k based on latency demand. The approximate space complexity can be calculated as = k×(ϱ k + 2ϱ k )×β = 3k•ϱ k •β The complexity of STOA models are defined like O n 3 +O (nlog 2 n) for Static scheme [14] , O n 2 + n for Game-based [12], O n 3 + O (n) for GORA [19], O n 2 + O (n) for EMSRA [25], respectively.…”

Aggregated Boolean Query Processing for Document Retrieval in Edge Computing

“…Traditional offloading schemes utilize heuristic algorithms to solve different optimization objectives, including network delay and energy consumption [32,33]. Intelligent offloading schemes solve the network delay and energy consumption problem, using the technique of online learning [34][35][36].…”