An Intelligent Offloading System Based on Multiagent Reinforcement Learning

Abstract

Intelligent vehicles have provided a variety of services; there is still a great challenge to execute some computing-intensive applications. Edge computing can provide plenty of computing resources for intelligent vehicles, because it offloads complex services from the base station (BS) to the edge computing nodes. Before the selection of the computing node for services, it is necessary to clarify the resource requirement of vehicles, the user mobility, and the situation of the mobile core network; they will affect the users’ quality of experience (QoE). To maximize the QoE, we use multiagent reinforcement learning to build an intelligent offloading system; we divide this goal into two suboptimization problems; they include global node scheduling and independent exploration of agents. We apply the improved Kuhn–Munkres (KM) algorithm to node scheduling and make full use of existing edge computing nodes; meanwhile, we guide intelligent vehicles to the potential areas of idle computing nodes; it can encourage their autonomous exploration. Finally, we make some performance evaluations to illustrate the effectiveness of our constructed system on the simulated dataset.

Multi-Agent DRL-Based Task Offloading in Multiple RIS-Aided IoV Networks

“…Accordingly, if n(V t ) ≤ n(agent), one agent will correspond to one V t , whereas if n(V t ) > n(agent), V t with similar parameters will correspond to the same agent. Such V t s with similar parameters are clustered for the same agent using k-means algorithm considering relative parameters such as speed, location, direction, etc., as measures for feature scaling [5], [26], [31]. Algorithm 1 illustrates the k-means algorithm used for vehicle clustering.…”

“…Each actor of every agent receives this packet at the beginning of a state and updates the packet by adding their own observed information and actions in the current state. This data packet then gets updated from f t−1 to f t , which contains the information of actor act t and action a t of all the agents at time t. Thus, the decision made by every agent is dependent upon the current and previous state of its own as well as other agents, which allow the agents to take global decisions [5]. • Reward: According to the observations, the agents take action a i t and receives reward r i t .…”

“…6) Mean delay for the tasks offloaded to BS: In Fig. 11, we show the mean delay for only tasks which are offloaded to the BS when the maximum tolerable delay lies within the range [5][6][7][8][9]. Here, we compare the mean delay when with and without the presence of RIS.…”

“…However, the onboard resources of a lowcost consumer vehicle may not be able to accommodate the computing needs of a vehicular fog computing (VFC) network, thereby affecting the quality of experience (QoE) of the fog vehicular network. Accordingly, to meet the requirement of such networks researchers have introduced multi-access edge computing (MAEC) which can extend the computation power of a vehicular network by allowing the vehicles in an IoV network to offload part of their task to the MAEC server located on the roadside or at the BS [3]- [5].…”

“…Accordingly, while in [25], the authors proposed a multi-agent DRL-based approach for a multi-tier framework, the authors in [26] proposed a multi-agent DRL approach to provide an optimal task offloading solution and minimize the task processing delay. Further, the authors in [27] proposed a multi-agent DRL-based approach for device-to-device communication, and likewise in [5], the authors formulated a multi-agent reinforcement learning-based algorithm for optimal offloading by using the improved Kuhn-Munkres (KM) algorithm for task scheduling. Although several DRLbased algorithms for MAEC and RIS-aided networks have been studied, researchers have not yet used multi-agent DRLbased algorithms in the domains of MAEC and RIS systems.…”