Dynamic Task Offloading Strategy for Multi-Agent Deep Reinforcement Learning Based on Lyapunov

Abstract

Aiming at the issue of communication overload and task backlog caused by the dynamic nature of the environment and the surge in the number of mobile devices in multi-user Mobile Edge Computing scenarios, a dynamic task offloading strategy based on Lyapunov-guided multi-agent deep reinforcement learning is proposed. This strategy aims to ensure long-term system stability while minimizing the average task processing latency and energy consumption. First, the dependency relationships among subtasks are modeled using a directed acyclic graph, and this is expressed as an optimization problem to minimize task offloading costs with long-term constraints. Then, using Lyapunov optimization theory, the long-term average problem is decoupled into deterministic problems for each time slot. Finally, the issue is further transformed into an optimal policy problem under a Markov Decision Process framework, and solved using the designed Lyapunov Multi-Agent Deep Deterministic Policy Gradient (Ly-MADDPG) algorithm. The simulation experiment results indicate that compared to the alternative algorithms, our proposed method reduces the task offloading cost while ensuring queue stability.