Learning-Based Queuing Delay-Aware Task Offloading in Collaborative Vehicular Networks

Jia, Zehan; Zhou, Zhenyu; Wang, Xiaoyan; Mumtaz, Shahid

doi:10.1109/icc42927.2021.9500852

Cited by 19 publications

(5 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Mirza et al [20] focus on DRL-assisted delay-optimized task offloading in Automotive-Industry 5.0 based Vehicular Edge Computing Networks (VECNs), addressing the nuances of delay-sensitive applications. Jia et al [21] introduce a learning-based queuing delay-aware task offloading approach, showcasing the adaptability of RL in addressing temporal considerations. Luo et al [22] contribute by minimizing the delay and cost of computation offloading in vehicular edge computing, emphasizing efficiency.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Intelligent Vehicle Computation Offloading in Vehicular Ad Hoc Networks: A Multi-Agent LSTM Approach with Deep Reinforcement Learning

Sun,

Chen,

2024

Mathematics

View full text Add to dashboard Cite

As distributed computing evolves, edge computing has become increasingly important. It decentralizes resources like computation, storage, and bandwidth, making them more accessible to users, particularly in dynamic Telematics environments. However, these environments are marked by high levels of dynamic uncertainty due to frequent changes in vehicle location, network status, and edge server workload. This complexity poses substantial challenges in rapidly and accurately handling computation offloading, resource allocation, and delivering low-latency services in such a variable environment. To address these challenges, this paper introduces a “Cloud–Edge–End” collaborative model for Telematics edge computing. Building upon this model, we develop a novel distributed service offloading method, LSTM Muti-Agent Deep Reinforcement Learning (L-MADRL), which integrates deep learning with deep reinforcement learning. This method includes a predictive model capable of forecasting the future demands on intelligent vehicles and edge servers. Furthermore, we conceptualize the computational offloading problem as a Markov decision process and employ the Multi-Agent Deep Deterministic Policy Gradient (MADDPG) approach for autonomous, distributed offloading decision-making. Our empirical results demonstrate that the L-MADRL algorithm substantially reduces service latency and energy consumption by 5–20%, compared to existing algorithms, while also maintaining a balanced load across edge servers in diverse Telematics edge computing scenarios.

show abstract

Section: Related Workmentioning

confidence: 99%

“…Each agent is tasked with updating its parameter set θ n in pursuit of the optimal policy π * θ n = argmax θ n J(θ n ). This involves maximizing its objective function J(θ n ), which is the agent's reward function, formulated as per Equation (21).…”

Section: Algorithm 1 Trajectory Prediction and Workload Prediction Al...mentioning

confidence: 99%

Intelligent Vehicle Computation Offloading in Vehicular Ad Hoc Networks: A Multi-Agent LSTM Approach with Deep Reinforcement Learning

Sun,

Chen,

2024

Mathematics

View full text Add to dashboard Cite

show abstract

“…There have been a number of problems with optimizing task offloading in highly dynamic vehicle networks, including insufficient information, conflicting queuing latency, and high dimensional curse. In [109], a queuing delay-aware task offloading algorithm based on DRL methods was proposed to dynamically improve the task offloading problem and maximize the throughput of user vehicles while meeting the requirements for the longterm queueing delay in collaborative vehicular networks. The simulation results indicate that the proposed method outperforms the D-QLOA and EMM approaches in terms of throughput and end-to-end queuing delay.…”

Section: ) Deep Q Network (Dqn)mentioning

confidence: 99%

Offloading Mechanisms Based on Reinforcement Learning and Deep Learning Algorithms in the Fog Computing Environment

Abdulazeez

Askar

2023

IEEE Access

View full text Add to dashboard Cite

Fog computing has emerged as a computing paradigm for resource-restricted Internet of things (IoT) devices to support time-sensitive and computationally intensive applications. Offloading can be utilized to transfer resource-intensive tasks from resource-limited end devices to a resource-rich fog or cloud layer to reduce end-to-end latency and enhance the performance of the system. However, this advantage is still challenging to achieve in systems with a high request rate because it leads to long queues of tasks in fog nodes and reveals inefficiencies in terms of delays. In this regard, reinforcement learning (RL) is a well-known method for addressing such decision-making issues. However, in large-scale wireless networks, both action and state spaces are complex and extremely extensive. Consequently, reinforcement learning techniques may not be able to identify an efficient strategy within an acceptable time frame. Hence, deep reinforcement learning (DRL) was developed to integrate RL and deep learning (DL) to address this problem. This paper presents a systematic analysis of using RL or DRL algorithms to address offloadingrelated issues in fog computing. First, the taxonomy of fog computing offloading mechanisms based on RL and DRL algorithms was divided into three major categories: value-based, policy-based, and hybridbased algorithms. These categories were then compared based on important features, including offloading problem formulation, utilized techniques, performance metrics, evaluation tools, case studies, their strengths and drawbacks, offloading directions, offloading mode, SDN-based architecture, and offloading decisions. Finally, the future research directions and open issues are discussed thoroughly.INDEX TERMS Fog computing, Internet of Things (IoT), offloading, reinforcement learning, deep reinforcement learning.

show abstract

“…DQN includes a main network and a target network, where the main network draws actions and optimizes strategy and the target network assists in training the main network. The parameters of main network and target network are denoted as 𝜔 main (t ) and 𝜔 target (t ), respectively [28,29]. PDAC is sum-…”

Section: Pdac Algorithmmentioning

confidence: 99%

\omega ^{\text{main}}(t)

and

\omega ^{\text{target}}(t)

, respectively [28, 29]. PDAC is summarized in Algorithm 1, which consists of three phases that is, initialization, access management, and model updating.…”

Section: Priority‐aware Dqn‐based Intelligent Access Managementmentioning

confidence: 99%

Priority‐aware intelligent device access management for carbon footprint monitoring in sustainable cites and society

Wang

Liao

et al. 2022

IET Communications

Self Cite

View full text Add to dashboard Cite

Carbon footprint monitoring provides significant basis for computing facilities to improve the computing efficiency and reduce the energy cost, which can enable low/zero carbon computing and facilitate the construction of sustainable cites. Massive sensing devices access to 5G base station to transmit collected carbon emission data, which requires intelligent access management. Fast uplink grant possesses the advantages of reducing signalling overhead and access conflicts while facing the problems of incomplete information and difficulty in guaranteeing priority constraint. This paper examines, the maximum access queuing delay minimization problem is formulated under the long‐term service priority constraint and the short‐term access management constraint. First, Lyapunov optimization is leveraged to decouple the long‐term service priority constraint and short‐term access management optimization, and decompose the long‐term stochastic optimization problem into a series of short‐term deterministic problems. Then, a priority‐aware deep Q‐network (DQN)‐based fast uplink grant access management (PDAC) algorithm is proposed to achieve intelligent access management with differentiated service priority requirements. PDAC utilizes DQN to handle non‐convex high‐dimensional optimization problem with service priority constraint to achieve intelligent access management and priority awareness. Simulation results demonstrate that PDAC outperforms the existing algorithms in access queuing delay, buffer queue backlog, and priority deficit fluctuation.

show abstract

Learning-Based Queuing Delay-Aware Task Offloading in Collaborative Vehicular Networks

Cited by 19 publications

References 18 publications

Intelligent Vehicle Computation Offloading in Vehicular Ad Hoc Networks: A Multi-Agent LSTM Approach with Deep Reinforcement Learning

Intelligent Vehicle Computation Offloading in Vehicular Ad Hoc Networks: A Multi-Agent LSTM Approach with Deep Reinforcement Learning

Offloading Mechanisms Based on Reinforcement Learning and Deep Learning Algorithms in the Fog Computing Environment

Priority‐aware intelligent device access management for carbon footprint monitoring in sustainable cites and society

Contact Info

Product

Resources

About