Adaptive Service Function Chain Scheduling in Mobile Edge Computing via Deep Reinforcement Learning

Abstract

MEC (Mobile Edge Computing) provides both IT service environment and cloud computation on the edge of the network. This technology not only minimizes the end-to-end latency but also increases the efficiency of computing. Some latency-sensitive applications, such as cloud video, online game, and augmented reality, take advantage of the MEC system to provide fast and stable services. Several new network techniques, including the implementation of NFV (Network Function Virtualization), the placement of VNF (Virtual Network Function) and the scheduling of SFC (Service Function Chain), should be considered to be applied in the MEC system. In this paper, we focus on the research about the scheduling of SFC in the NFV enabled MEC system and propose a solution accordingly. First, we make reasonable assumptions on the settings of MEC systems and model the SFC scheduling problem into a flexible job-shop scheduling problem. Since minimizing the latency can significantly improve the quality of service (QoS) and increase the revenue of Internet Service Providers, our optimization goal is to minimize the overall scheduling latency. To solve this optimization problem, a deep reinforcement learning based algorithm DQS is proposed. DQS can detect the variation of the MEC system's environment and perform adaptive scheduling for SFC requests. As the results of the simulation indicate, DQS works better than the other off-the-shelf algorithms in two key indexes: overall scheduling latency and average resource usage. Moreover, DQS can shorten the decision time and schedule SFCs stably with high performance. It is suitable to be extended to an online scheduling algorithm.

Reinforcement Learning-Empowered Mobile Edge Computing for 6G Edge Intelligence

“…In MEC, service deployment at distributed servers is complicated by dynamics from free mobility and wireless channels. In [50], [51], [130], [131], with the aid of virtualized technology (e.g., SDN and NFV), service deployment was discussed, as shown in Table V. In [130], for an MEC system with multiple virtual network function (VNF) components, the SFC scheduling problem was formulated as the job-shop scheduling problem.…”

“…In [50], [51], [130], [131], with the aid of virtualized technology (e.g., SDN and NFV), service deployment was discussed, as shown in Table V. In [130], for an MEC system with multiple virtual network function (VNF) components, the SFC scheduling problem was formulated as the job-shop scheduling problem. To complete a specific mobile device task, where to place the VNF and how to transmit a series of network function instances between microdata centers complicate conventional optimization.…”

Service Function Chaining in MEC: A Mean-Field Game and Reinforcement Learning Approach

“…The VNF/SFC resource allocation approaches can be grouped into three main categories, theoretical games approaches [9], [35], linear programming variant approaches [10]- [12], the machine learning approaches [13], [14] and the heuristic approaches [15], [16].…”

“…However, the request arrival rate in this work does not considered stochastic and the RL is based on Q-Learning algorithm, which acts only on a set of specific fixed instance of the problem. Work in [14] models the SFC scheduling problem as a flexible job-shop scheduling problem with the objective to minimize the scheduling latency. Authors proposed a deep RL based on Q-Learning that gives the environment the advantage of performing adaptive scheduling.…”

“…The two subproblems of VNFs placement and chaining are often addressed jointly in the literature [9]- [16] and many approaches have been proposed for this purpose. We can classify these approaches as heuristic approaches [17], game theory-based approaches [9], machine learning-based approaches [14] and linear programming-based approaches [11]. However, these works concentrate on cost minimization under service level agreement constraints without considering MEC constraints.…”