A Deep Reinforcement Learning-Based Resource Management Game in Vehicular Edge Computing

Zhu, Xiaoyu; Luo, Yueyi; Liu, Anfeng; Xiong, Neal N.; Dong, Mianxiong; Zhang, Shaobo

doi:10.1109/tits.2021.3114295

Cited by 56 publications

(12 citation statements)

References 43 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The DRL has achieved profound impacts recently [22], [23]. A series of relevant methods have been proposed and extensively applied in various fields, such as robot control [24], [25], automatic driving [26], edge computing [27], traffic control [28] and others. Specifically, the agent of DRL repeatedly explores the environment and maximizes reward to get excellent performance.…”

Section: A Multi-agent Reinforcement Learningmentioning

confidence: 99%

Multi-Agent Broad Reinforcement Learning for Intelligent Traffic Light Control

Zhu¹,

Li²,

Wu³

et al. 2022

Preprint

View full text Add to dashboard Cite

Intelligent Traffic Light Control System (ITLCS) is a typical Multi-Agent System (MAS), which comprises multiple roads and traffic lights. Constructing a model of MAS for ITLCS is the basis to alleviate traffic congestion. Existing approaches of MAS are largely based on Multi-Agent Deep Reinforcement Learning (MADRL). Although the Deep Neural Network (DNN) of MABRL is effective, the training time is long, and the parameters are difficult to trace. Recently, Broad Learning Systems (BLS) provided a selective way for learning in the deep neural networks by a flat network. Moreover, Broad Reinforcement Learning (BRL) extends BLS in Single Agent Deep Reinforcement Learning (SADRL) problem with promising results. However, BRL does not focus on the intricate structures and interaction of agents. Motivated by the feature of MADRL and the issue of BRL, we propose a Multi-Agent Broad Reinforcement Learning (MABRL) framework to explore the function of BLS in MAS. Firstly, unlike most existing MADRL approaches, which use a series of deep neural networks structures, we model each agent with broad networks. Then, we introduce a dynamic self-cycling interaction mechanism to confirm the "3W" information: When to interact, Which agents need to consider, What information to transmit. Finally, we do the experiments based on the intelligent traffic light control scenario. We compare the MABRL approach with six different approaches, and experimental results on three datasets verify the effectiveness of MABRL.

show abstract

Section: A Multi-agent Reinforcement Learningmentioning

confidence: 99%

Multi-Agent Broad Reinforcement Learning for Intelligent Traffic Light Control

Zhu¹,

Li²,

Wu³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…As artificial intelligence has advanced, reinforcement learning (RL) has become an important class of algorithms in artificial intelligence, and applications based on reinforcement learning have achieved effective results in edge computing [ 13 ], resource allocation [ 14 ], routing decisions [ 15 , 16 , 17 ], and so forth. Reinforcement learning is a methodological framework for learning, predicting, and making decisions.…”

Section: Introductionmentioning

confidence: 99%

Delay-Aware and Link-Quality-Aware Geographical Routing Protocol for UANET via Dueling Deep Q-Network

Zhang

Qiu

2023

Sensors

View full text Add to dashboard Cite

In an unmanned aerial vehicles ad hoc network (UANET), UAVs communicate with each other to accomplish intricate tasks collaboratively and cooperatively. However, the high mobility of UAVs, the variable link quality, and heavy traffic loads can lead to difficulties in finding an optimal communication path. We proposed a delay-aware and link-quality-aware geographical routing protocol for a UANET via the dueling deep Q-network (DLGR-2DQ) to address these problems. Firstly, the link quality was not only related to the physical layer metric, the signal-to-noise ratio, which was influenced by path loss and Doppler shifts, but also the expected transmission count of the data link layer. In addition, we also considered the total waiting time of packets in the candidate forwarding node in order to decrease the end-to-end delay. Then, we modeled the packet-forwarding process as a Markov decision process. We crafted an appropriate reward function that utilized the penalty value for each additional hop, total waiting time, and link quality to accelerate the learning of the dueling DQN algorithm. Finally, the simulation results illustrated that our proposed routing protocol outperformed others in terms of the packet delivery ratio and the average end-to-end delay.

show abstract

“…Moreover, considering the limited resources of VEC servers, there exist some works that target on efficient resource allocation for VEC servers [10], [11], [12]. For example, Peng et al [10] employ a deep learning approach to manage the resources of VEC servers for the delay-sensitive applications of vehicles.…”

Section: Introductionmentioning

confidence: 99%

“…For example, Peng et al [10] employ a deep learning approach to manage the resources of VEC servers for the delay-sensitive applications of vehicles. Zhu et al [11] adopt a Stackelberg game to model the interaction between vehicles and VEC servers to obtain the price and amount of computation resources to be allocated. In [12], the authors focus on the power-aware resource management to jointly optimize the resource utilization and energy efficiency of VEC servers.…”

Section: Introductionmentioning

confidence: 99%

BARGAIN-MATCH: A Game Theoretical Approach for Resource Allocation and Task Offloading in Vehicular Edge Computing Networks

Sun¹

2022

Preprint

View full text Add to dashboard Cite

Vehicular edge computing (VEC) is emerging as a promising architecture of vehicular networks (VNs) by deploying the cloud computing resources at the edge of the VNs. This work aims to optimize resource allocation and task offloading in VEC networks. Specifically, we formulate a game theoretical resource allocation and task offloading problem (GTRATOP) that aims to maximize the system performance by jointly considering the incentive for cooperation, competition among vehicles, heterogeneity between VEC servers and vehicles, and inherent dynamic of VNs. Since the formulated GTRATOP is NP-hard, we propose an adaptive approach for resource allocation and task offloading in VEC networks by incorporating bargaining game and matching game, which is called BARGAIN-MATCH. First, for resource allocation, a bargaining game-based incentive is proposed to stimulate the vehicles and VEC servers to negotiate the optimal resource allocation and pricing decisions. Second, for task offloading, a many-to-one matching scheme is proposed to decide the optimal offloading strategies. Third, the dynamic and time-varying features are considered to adapt the strategies of BARGAIN-MATCH to the real-time VEC networks. Moreover, the BARGAIN-MATCH is proved to be stable and weak Pareto optimal. Simulation results demonstrate that the proposed BARGAIN-MATCH achieves superior system performance and efficiency compared to other methods, especially when the system workload is heavy.

show abstract

A Deep Reinforcement Learning-Based Resource Management Game in Vehicular Edge Computing

Cited by 56 publications

References 43 publications

Multi-Agent Broad Reinforcement Learning for Intelligent Traffic Light Control

Multi-Agent Broad Reinforcement Learning for Intelligent Traffic Light Control

Delay-Aware and Link-Quality-Aware Geographical Routing Protocol for UANET via Dueling Deep Q-Network

BARGAIN-MATCH: A Game Theoretical Approach for Resource Allocation and Task Offloading in Vehicular Edge Computing Networks

Contact Info

Product

Resources

About