Online Service Migration in Mobile Edge with Incomplete System Information: A Deep Recurrent Actor-Critic Learning Approach

Abstract

Multi-access Edge Computing (MEC) is an emerging computing paradigm that extends cloud computing to the network edge to support resource-intensive applications on mobile devices. As a crucial problem in MEC, service migration needs to decide how to migrate user services for maintaining Quality-of-Service when users roam between MEC servers with limited coverage and capacity. However, finding an optimal migration policy is intractable due to the dynamic MEC environment and user mobility. Many existing works make centralized migration decisions based on complete system-level information, which can be time-consuming and also lack good scalability. To address these challenges, we propose a novel learning-driven method, which is user-centric and makes effective online migration decisions by utilizing incomplete system-level information. Specifically, the service migration problem is modeled as a Partially Observable Markov Decision Process (POMDP). To solve the POMDP, we design a new encoder network that combines a Long Short-Term Memory (LSTM) and an embedding matrix for effective extraction of hidden information, and propose a tailored off-policy actor-critic algorithm for efficient training. The extensive experimental results based on real-world mobility traces demonstrate that our method consistently outperforms both the heuristic and state-of-the-art learning-driven algorithms, and can achieve near-optimal results on various MEC scenarios.

A Two-Timescale Approach to Mobility Management for Multicell Mobile Edge Computing

“…Note that the transmit power still is bounded by (17). We obtain for this case the migration decision relies on two parameters H n (k) and h min n (k), in which h min n (k) [see (19)] is monotonically decreasing with f n (k).…”

“…The last approach, which is closely related to this work, focuses on online migration design without a priori knowledge of future user mobility. Specifically, learning-driven migration schemes are proposed in [17], [18] based on multi-armed bandit theory, and in [19] using the deep reinforcement learning approach, in which the user copes with the lack of prior knowledge using the trial-and-error method. On the other hand, by utilizing the Lyapunov optimization technique, an online migration strategy is proposed in [20] that balances the service latency, the incurred migration cost, and the long-term user movement.…”

“…Proposition 3 shows that for this case, the computation rate affects the migration decision through the minimum channel threshold h min n [see (19)], with h min n being decreasing as f n (k)…”

Service migration in MEC: an Approach Based on Deep Reinforcement Learning and Fuzzy Logic

“…We investigate the MEC case with multiple nodes and a large number of access users. Each MEC node has a BS that cooperates to provide wireless connectivity to users according to [31]. The users move within the coverage of the BS and obtain resources through the BS.…”

A Distributed Learning Architecture for Semantic Communication in Autonomous Driving Networks for Task Offloading

“…Such a system supports the provisioning of resource-intensive applications in autonomous vehicles [1]. VEC provides computational resources to the vehicles thereby reducing service latency and energy consumption [2] while significantly improving the Quality-of-Service (QoS) of the ADNs.…”