Scheduling for Mobile Edge Computing With Random User Arrivals—An Approximate MDP and Reinforcement Learning Approach

Huang, Shanfeng; Lv, Bojie; Wang, Rui; Huang, Kaibin

doi:10.1109/tvt.2020.2990482

Cited by 25 publications

(15 citation statements)

References 29 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In the scenario that APs and edge servers are connected via software defined network (SDN), the authors in [5] proposed a heuristic algorithm to dispatch the jobs to the closest edge servers according to their locations. Considering random jobs arrival and job offloading to a single edge server, the authors in [11], [12] formulate the offloading problem as an infinite-horizon Markov decision process (MDP). In the above works, a centralized dispatcher with complete and updated knowledge of the system states was assumed in the edge computing systems, which might be impractical.…”

Section: Related Workmentioning

confidence: 99%

“…Pr S(t + 1) S(t), Ω(S(t), D(t)) • V S(t + 1) , (12) where the value function V (S(t)) of the optimal policy Ω * is defined as follows.…”

Section: Pomdp-based Problem Formulationmentioning

confidence: 99%

“…In this section, we shall introduce a novel approximation method to decouple the centralized optimization on the RHS of the Bellman's equations in equation (12) to each AP for arbitrary system state. For the APs outside the conflict AP sets of each other, the update of dispatching actions at one AP will not affect the task computation originated from other APs.…”

Section: Distributed Algorithm With Partial Informationmentioning

confidence: 99%

See 2 more Smart Citations

Online Distributed Job Dispatching with Outdated and Partially-Observable Information

Hong

Wang

et al. 2020

Preprint

Self Cite

View full text Add to dashboard Cite

In this paper, we investigate online distributed job dispatching in an edge computing system residing in a Metropolitan Area Network (MAN). Specifically, job dispatchers are implemented on access points (APs) which collect jobs from mobile users and distribute each job to a server at the edge or the cloud. A signaling mechanism with periodic broadcast is introduced to facilitate cooperation among APs. The transmission latency is non-negligible in MAN, which leads to outdated information sharing among APs. Moreover, the fully-observed system state is discouraged as reception of all broadcast is time consuming. Therefore, we formulate the distributed optimization of job dispatching strategies among the APs as a Markov decision process with partial and outdated system state, i.e., partially observable Markov Decision Process (POMDP). The conventional solution for POMDP is impractical due to huge time complexity. We propose a novel low-complexity solution framework for distributed job dispatching, based on which the optimization of job dispatching policy can be decoupled via an alternative policy iteration algorithm, so that the distributed policy iteration of each AP can be made according to partial and outdated observation. A theoretical performance lower bound is proved for our approximate MDP solution. Furthermore, we conduct extensive simulations based on the Google Cluster trace. The evaluation results show that our policy can achieve as high as 20.67% reduction in average job response time compared with heuristic baselines, and our algorithm consistently performs well under various parameter settings.

show abstract

Section: Related Workmentioning

confidence: 99%

“…Pr S(t + 1) S(t), Ω(S(t), D(t)) • V S(t + 1) , (12) where the value function V (S(t)) of the optimal policy Ω * is defined as follows.…”

Section: Pomdp-based Problem Formulationmentioning

confidence: 99%

Section: Distributed Algorithm With Partial Informationmentioning

confidence: 99%

See 1 more Smart Citation

Online Distributed Job Dispatching with Outdated and Partially-Observable Information

Hong

Wang

et al. 2020

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…Edge computing is a hot research field in recent years, which sinks a large number of complex calculations into the edge server environment to reduce cloud burden and delay [5], thus making autonomous driving technology possible [6]. Edge computing has played an important role in autonomous driving [7][8][9], Internet of Things (IoT) [10,11], data privacy [12,13], and other research fields [14,15].…”

Section: Introductionmentioning

confidence: 99%

Optimize the Communication Cost of 5G Internet of Vehicles through Coherent Beamforming Technology

Shi

et al. 2021

Wireless Communications and Mobile Computing

View full text Add to dashboard Cite

Edge computing, which sinks a large number of complex calculations into edge servers, can effectively meet the requirement of low latency and bandwidth efficiency and can be conducive to the development of the Internet of Vehicles (IoV). However, a large number of edge servers mean a big cost, especially for the 5G scenario in IoV, because of the small coverage of 5G base stations. Fortunately, coherent beamforming (CB) technology enables fast and long-distance transmission, which gives us a possibility to reduce the number of 5G base stations without losing the whole network performance. In this paper, we try to adopt the CB technology on the IoV 5G scenario. We suppose we can arrange roadside nodes for helping transferring tasks of vehicles to the base station based on the CB technology. We first give the mathematical model and prove that it is a NP-hard model that cannot be solved directly. Therefore, we design a heuristic algorithm for an Iterative Coherent Beamforming Node Design (ICBND) algorithm to obtain the approximate optimal solution. Simulation results show that this algorithm can greatly reduce the cost of communication network infrastructure.

show abstract

“…Therefore, researchers proposed using reinforcement learning (RL) to build a smart congestion control method. As a research hotspot in the machine learning field, reinforcement learning has been widely used in UAV control [5], robot control [6], optimization and scheduling [7,8] and computer games [9]. The basic idea of RL is to construct an agent to interact with the environment, and to learn the optimal strategy by maximizing the cumulative reward obtained by the agent from the environment.…”

Section: Introductionmentioning

confidence: 99%

An Intelligent TCP Congestion Control Method Based on Deep Q Network

Wang

Dong

2021

Future Internet

View full text Add to dashboard Cite

To optimize the data migration performance between different supercomputing centers in China, we present TCP-DQN, which is an intelligent TCP congestion control method based on DQN (Deep Q network). The TCP congestion control process is abstracted as a partially observed Markov decision process. In this process, an agent is constructed to interact with the network environment. The agent adjusts the size of the congestion window by observing the characteristics of the network state. The network environment feeds back the reward to the agent, and the agent tries to maximize the expected reward in an episode. We designed a weighted reward function to balance the throughput and delay. Compared with traditional Q-learning, DQN uses double-layer neural networks and experience replay to reduce the oscillation problem that may occur in gradient descent. We implemented the TCP-DQN method and compared it with mainstream congestion control algorithms such as cubic, Highspeed and NewReno. The results show that the throughput of TCP-DQN can reach more than 2 times of the comparison method while the latency is close to the three compared methods.

show abstract

Scheduling for Mobile Edge Computing With Random User Arrivals—An Approximate MDP and Reinforcement Learning Approach

Cited by 25 publications

References 29 publications

Online Distributed Job Dispatching with Outdated and Partially-Observable Information

Online Distributed Job Dispatching with Outdated and Partially-Observable Information

Optimize the Communication Cost of 5G Internet of Vehicles through Coherent Beamforming Technology

An Intelligent TCP Congestion Control Method Based on Deep Q Network

Contact Info

Product

Resources

About