Deep Reinforcement Learning for Offloading and Resource Allocation in Vehicle Edge Computing and Networks

Liu, Yi; Yu, Haozheng; Xie, Shengli; Zhang, Yan

doi:10.1109/tvt.2019.2935450

Cited by 441 publications

(151 citation statements)

References 54 publications

Supporting

Mentioning

150

Contrasting

Unclassified

Order By: Relevance

“…Experimental results show that the proposed algorithm performs better than the MUMTO algorithm in [177] in terms of overall cost, energy consumption, and delay. Liu et al [144] formulated offloading and resource allocation in VECNs as a semi-Markov process, by considering stochastic vehicle traffic, dynamic computation requests, and time-varying communication conditions. Two RL methods, i.e., a Q-learning based method and a DRL method, are designed to get the optimal policies for computation offloading and resource allocation.…”

Section: ) Lbs For Joint Issuesmentioning

confidence: 99%

A Review on Computational Intelligence Techniques in Cloud and Edge Computing

Asim

Wang

et al. 2020

IEEE Trans. Emerg. Top. Comput. Intell.

View full text Add to dashboard Cite

Cloud computing (CC) is a centralized computing paradigm that accumulates resources centrally and provides these resources to users through Internet. Although CC holds a large number of resources, it may not be acceptable by real-time mobile applications, as it is usually far away from users geographically. On the other hand, edge computing (EC), which distributes resources to the network edge, enjoys increasing popularity in the applications with low-latency and high-reliability requirements. EC provides resources in a decentralized manner, which can respond to users' requirements faster than the normal CC, but with limited computing capacities. As both CC and EC are resource-sensitive, several big issues arise, such as how to conduct job scheduling, resource allocation, and task offloading, which significantly influence the performance of the whole system. To tackle these issues, many optimization problems have been formulated. These optimization problems usually have complex properties, such as non-convexity and NP-hardness, which may not be addressed by the traditional convex optimization-based solutions. Computational intelligence (CI), consisting of a set of nature-inspired computational approaches, recently exhibits great potential in addressing these optimization problems in CC and EC. This paper provides an overview of research problems in CC and EC and recent progresses in addressing them with the help of CI techniques. Informative discussions and future research trends are also presented, with the aim of offering insights to the readers and motivating new research directions.

show abstract

Section: ) Lbs For Joint Issuesmentioning

confidence: 99%

A Review on Computational Intelligence Techniques in Cloud and Edge Computing

Asim

Wang

et al. 2020

IEEE Trans. Emerg. Top. Comput. Intell.

View full text Add to dashboard Cite

show abstract

“…With the growth of cloud computing applications, mobile edge computing (MEC) has seen broad application prospects [28]. In the area of MEC task offloading, Liu et al [29] considered the dynamic changes of user equipment (UE) location and service requests and proposed a DRL-based task scheduling scheme. Large numbers of vehicles are used as mobile edge servers to provide computing services to nearby equipment, effectively solving the task scheduling problem in a changing environment.…”

Section: A Research On Job Schedulingmentioning

confidence: 99%

A Two-Stage Framework for the Multi-User Multi-Data Center Job Scheduling and Resource Allocation

et al. 2020

View full text Add to dashboard Cite

With the development of artificial intelligence and the Internet of things, the prospects of cloud computing applications have become broader, and the number of users and cloud data centers (CDCs) has exploded. It is a challenge to realize efficient job scheduling and resource allocation of multiple users and data centers. However, the traditional scheduling model based on heuristic algorithm has some limitations in the complex and changeable cloud environment. In addition, many existing single-agent models rarely consider the multi-objective global optimization of the system. Therefore, this paper proposes a two-stage job scheduling and resource allocation framework that adopts multiple intelligent schedulers to solve the cooperative scheduling problem between job scheduling and resource allocation. A heterogeneous distributed deep learning (HDDL) model is used in the job scheduling stage to schedule multiple jobs to multiple cloud data centers. The deep Q-network (DQN) model is a resource scheduler to deploy virtual machine to physical servers for execution. Extensive numerical results show that both HDDL-baesd job scheduler and DQN-based resource allocator outperform the benchmark algorithm in terms of optimizing energy consumption and job delay. Furthermore, the proposed framework not only can achieve a global near-optimum by achieving local optimization at each stage but also has good scalability and low computation delay.

show abstract

“…Nevertheless, by exploiting deep neural networks (DNNs) for function approximation, deep reinforcement learning (DRL) has been demonstrated to be able to efficiently approximate Q-values of RL [22] and more scalable. There have been some attempts to adopt DRL in the design of online resource allocation and scheduling for computation offloading in MEC [23][24][25][26][27]. Specifically, in [23], system sum cost of a multi-user network is minimized in terms of execution delay and energy consumption by computational resource allocation.…”

Section: Introductionmentioning

confidence: 99%

“…Besides, double deep Q-network (DQN)-based strategic computation offloading algorithm was proposed in [26], where an mobile device learned the optimal task offloading and energy allocation to maximize the long-term utility based on the task queue state, the energy queue state as well as the channel qualities. What is more, a DQN-based vehicle-assisted offloading scheme is studied in [27] to maximize the long-term utility of the vehicle edge computing network by considering the delay of the computation task. Overall, existing works on DRL-based dynamic computation offloading only consider centralized algorithms for either single user cases or multi-user scenarios.…”

Section: Introductionmentioning

confidence: 99%

Decentralized computation offloading for multi-user mobile edge computing: a deep reinforcement learning approach

Zhao

Wang

2020

J Wireless Com Network

135

View full text Add to dashboard Cite

Mobile edge computing (MEC) emerges recently as a promising solution to relieve resource-limited mobile devices from computation-intensive tasks, which enables devices to offload workloads to nearby MEC servers and improve the quality of computation experience. In this paper, an MEC enabled multi-user multi-input multi-output (MIMO) system with stochastic wireless channels and task arrivals is considered. In order to minimize long-term average computation cost in terms of power consumption and buffering delay at each user, a deep reinforcement learning (DRL)-based dynamic computation offloading strategy is investigated to build a scalable system with limited feedback. Specifically, a continuous action space-based DRL approach named deep deterministic policy gradient (DDPG) is adopted to learn decentralized computation offloading policies at all users respectively, where local execution and task offloading powers will be adaptively allocated according to each user’s local observation. Numerical results demonstrate that the proposed DDPG-based strategy can help each user learn an efficient dynamic offloading policy and also verify the superiority of its continuous power allocation capability to policies learned by conventional discrete action space-based reinforcement learning approaches like deep Q-network (DQN) as well as some other greedy strategies with reduced computation cost. Besides, power-delay tradeoff for computation offloading is also analyzed for both the DDPG-based and DQN-based strategies.

show abstract

Deep Reinforcement Learning for Offloading and Resource Allocation in Vehicle Edge Computing and Networks

Cited by 441 publications

References 54 publications

A Review on Computational Intelligence Techniques in Cloud and Edge Computing

A Review on Computational Intelligence Techniques in Cloud and Edge Computing

A Two-Stage Framework for the Multi-User Multi-Data Center Job Scheduling and Resource Allocation

Decentralized computation offloading for multi-user mobile edge computing: a deep reinforcement learning approach

Contact Info

Product

Resources

About