Task Offloading with Power Control for Mobile Edge Computing Using Reinforcement Learning-Based Markov Decision Process

Abstract

This paper proposes an efficient computation task offloading mechanism for mobile edge computing (MEC) systems. The studied MEC system consists of multiple user equipment (UEs) and multiple radio interfaces. In order to maximize the number of UEs benefitting from the MEC, the task offloading and power control strategy for a UE is optimized in a joint manner. However, the problem of finding the optimal solution is NP-hard. We then reformulate the problem as a Markov decision process (MDP) and develop a reinforcement learning- (RL-) based algorithm to solve the MDP. Simulation results show that the proposed RL-based algorithm achieves a near-optimal performance compared to the exhaustive search algorithm, and it also outperforms the received signal strength- (RSS-) based method no matter from the standpoint of the system (as it leads to a larger number of beneficial UEs) or an individual (as it generates a lower computation overhead for a UE).

Task Offloading in Autonomous IoT Systems using Deep Reinforcement Learning and ns3-gym

“…This approach is the one adopted also in [10], [22], [9], [25], [19], [31] and [20], as the authors have treated task offloading in conjunction with the problem of resource allocation in resource constrained networks. Task offloading can be approached as a Markov Decision Process (MDP) problem as suggested by the authors in [8], [34], [29], [32], [26], [7], [8], [33] and [15]. To determine the optimal policy for the MDP problem several works, [32], [26], [15], have implemented strategies based on Q-learning, a classical reinforcement learning algorithm.…”

“…Task offloading can be approached as a Markov Decision Process (MDP) problem as suggested by the authors in [8], [34], [29], [32], [26], [7], [8], [33] and [15]. To determine the optimal policy for the MDP problem several works, [32], [26], [15], have implemented strategies based on Q-learning, a classical reinforcement learning algorithm. Meanwhile this algorithm suffers from the same deficits of resolution time as the heuristic methods.…”

“…From an architectural point of view, the majority of the works ( [16], [22], [25], [6], [2], [19], [31], [7], [8], [33], [20], [15] and [1]) adopt a centralized architecture while some others [36], [10], [6], [30], [32], [26] have implemented a decentralized architecture. In the centralized architecture, offloading decisions are made at a central point, whereas in a decentralized architecture each IoT device implements the decision making algorithm.…”

“…This allows for different offloading scenarios [3]. A collaboration between IoT devices and an edge server is implemented in [34], [23], [16], [9], [25], [32], [2], [19], [31], [7], [8], [20], [15] and [1]. This is a vertical collaboration.…”

AutoFL: Enabling Heterogeneity-Aware Energy Efficient Federated Learning

“…For example, Gaudette et al proposed to use arbitrary polynomial chaos expansions to consider the effect of various uncertainties on mobile user experience [33]. Other computation offloading techniques also consider the performance variability aspect in the mobile environment for energy efficiency optimization [2,3,21,47,59,61,62,92,99,131,134]. While the aforementioned techniques addressed similar runtime variance in the edge-cloud execution environment, prior works are sub-optimal for FL because of the highly distributed nature of FL use cases-not only that system and data heterogeneity can easily degrade the quality of FL, but runtime variance can also introduce uncertainties in FL's training time performance and execution efficiency.…”