Resilient multi-agent RL: introducing DQ-RTS for distributed environments with data loss

Abstract

This paper proposes DQ-RTS, a novel decentralized Multi-Agent Reinforcement Learning algorithm designed to address challenges posed by non-ideal communication and a varying number of agents in distributed environments. DQ-RTS incorporates an optimized communication protocol to mitigate data loss between agents. A comparative analysis between DQ-RTS and its decentralized counterpart Q-RTS, or Q-learning for Real-Time Swarms, demonstrates the superior convergence speed of DQ-RTS, achieving a remarkable speed-up factor ranging from 1.6 to 2.7 in scenarios with non-ideal communication. Moreover, DQ-RTS exhibits robustness by maintaining performance even when the agent population fluctuates, making it well-suited for applications requiring adaptable agent numbers over time. Additionally, extensive experiments conducted on various benchmark tasks validate the scalability and effectiveness of DQ-RTS, further establishing its potential as a practical solution for resilient Multi-Agent Reinforcement Learning in dynamic distributed environments.

Design and Development of Multi-Agent Reinforcement Learning Intelligence on the Robotarium Platform for Embedded System Applications

“…Furthermore, Ref. [14]'s exploration of MARL led to the development of a novel approach, Decentralized Q-RTS, aimed at addressing challenges such as hardware implementation and achieving fully decentralized agent behavior. This innovative framework is particularly adept at handling scenarios where data transmission between agents fails or the number of available agents fluctuates, thereby ensuring robust operation in dynamic environments while facilitating the dissemination of knowledge among agents.…”

“…This surge in interest is due to RL's ability to tackle tasks in a manner akin to human cognitive processes. Currently, RL has found diverse applications across domains such as finance, robotics [10], natural language processing, and telecommunications [11][12][13][14]. In RL, the agent's performance is evaluated through a feedback mechanism called a reward (R t ).…”

Maze-solving in a plasma system based on functional analogies to reinforcement-learning model

“…As a maze solver, the heuristic property that RL substantially possesses is beneficial, and its feasibility can be upgraded in several ways. For instance, after the initial proposal of Q-learning [ 12 ], inclusion of multi-agent models and its decentralization system was recently proposed [ 19 ], which enhances rapid convergence with robustness. In the case of maze-solving in a simple RL model, the distribution of the reward spreading from the exit is a key issue, and its spatial pattern is configured in automatic and arbitrary iterations.…”