Deep Reinforcement Learning for Joint User Association and Resource Allocation in Factory Automation

Abstract

We propose joint user association, channel assignment and power allocation for mobile robot Ultra-Reliable and Low Latency Communications (URLLC) based on multiconnectivity and reinforcement learning. The mobile robots require control messages from the central guidance system at regular intervals. We use a two-phase communication scheme where robots can form multiple clusters. The robots in a cluster are close to each other and can have reliable Deviceto-Device (D2D) communications. In Phase I, the APs transmit the combined payload of a cluster to the cluster leader within a latency constraint. The cluster leader broadcasts this message to its members in Phase II. We develop a distributed Multi-Agent Reinforcement Learning (MARL) algorithm for joint user association and resource allocation (RA) for Phase I. The cluster leaders use their local Channel State Information (CSI) to decide the APs for connection along with the sub-band and power level. The cluster leaders utilize multi-connectivity to connect to multiple APs to increase their reliability. The objective is to maximize the successful payload delivery probability for all robots. Illustrative simulation results indicate that the proposed scheme can approach the performance of the centralized algorithm and offer a substantial gain in reliability as compared to single-connectivity (when cluster leaders are able to connect to 1 AP).

Leveraging RL for Efficient Collection of Perception Messages in Vehicular Networks

“…All the aforementioned challenges pushed us to conceive a solution that leverages artificial intelligence and machine learning [4]. Reinforcement learning (RL) techniques are a promising solution to address the challenges faced in vehicular networks [5], particularly regarding load balancing and network resource management. In this paper, we suggest a frequency-adjusting RL algorithm, that relies on smart clustering, to streamline the process of determining which message type would be most advantageous for transmission in vehicular networks.…”

Controlling optical-cavity locking using reinforcement learning

“…Reinforcement Learning (RL) algorithms [1] have emerged as a pivotal force reshaping the scientific landscape by offering unprecedented capabilities to learn optimal actions leading to eventual success in uncharted environments, all without the need for external supervision. This transformative paradigm has found application across a spectrum of domains, ranging from self-driving cars [2,3], industrial automation [4,5], trading and finance [6], healthcare [7], gaming [8,9], to the intricacies of optics [10][11][12].…”