On the Robustness of Cooperative Multi-Agent Reinforcement Learning

Lin, Jieyu; Dzeparoska, Kristina; Zhang, Sai Qian; Leon-Garcia, Alberto; Papernot, Nicolas

doi:10.1109/spw50608.2020.00027

Cited by 44 publications

(39 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Adversarial attacks for DRL agents. Most of existing adversarial attacks on DRL agents are on single agent [Huang et al, 2017, Lin et al, 2017, Kos and Song, 2017, Weng et al, 2019 while there is only one work [Lin et al, 2020] that focuses on the c-MARL setting, which is the most relevant work to our problem setting on attacking multi-agent RL.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Evaluating Robustness of Cooperative MARL: A Model-based Approach

Pham¹,

Nguyen²,

Chen³

et al. 2022

Preprint

View full text Add to dashboard Cite

In recent years, a proliferation of methods were developed for cooperative multi-agent reinforcement learning (c-MARL). However, the robustness of c-MARL agents against adversarial attacks has been rarely explored. In this paper, we propose to evaluate the robustness of c-MARL agents via a model-based approach. Our proposed formulation can craft stronger adversarial state perturbations of c-MARL agents(s) to lower total team rewards more than existing model-free approaches. In addition, we propose the first victim-agent selection strategy which allows us to develop even stronger adversarial attack. Numerical experiments on multi-agent MuJoCo benchmarks illustrate the advantage of our approach over other baselines. The proposed model-based attack consistently outperforms other baselines in all tested environments.

show abstract

Section: Related Workmentioning

confidence: 99%

“…However, there are two major differences between our work and Lin et al [2020]: (1) their attack is only evaluated under the StarCraft Multi-Agent Challenge (SMAC) environment [Samvelyan et al, 2019] where the action spaces are discrete;…”

Section: Related Workmentioning

confidence: 99%

Evaluating Robustness of Cooperative MARL: A Model-based Approach

Pham¹,

Nguyen²,

Chen³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…There is another work [9], which is focused on resilience in cooperative MAS and propose an Antagonist-Ratio Training Scheme (ARTS) by reformulating the original target MAS as a mixed cooperative-competitive game between a group of protagonists which represent agents of the target MAS and a group of antagonists which represent failures in the MAS. However, Lin [7] introduces a novel attack where the attacker first trains a policy network with reinforcement learning to find a wrong action it should encourage the victim agent to take. Then, the adversary uses targeted adversarial examples to force the victim to take this action.…”

Section: Related Workmentioning

confidence: 99%

Task-Based Ad-hoc Teamwork with Adversary

Shafipour

Fallah

2021

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Many real-world applications require agents to cooperate and collaborate to accomplish shared missions; though, there are many instances where the agents should work together without communication or prior coordination. In the meantime, agents often coordinate in a decentralised manner to complete tasks that are displaced in an environment (e.g., foraging, demining, rescue or firefighting). Each agent in the team is responsible for selecting their own task and completing it autonomously. However, there is a possibility of an adversary in the team, who tries to prevent other agents from achieving their goals. In this study, we assume there is an agent who estimates the model of other agents in the team to boost the team's performance regardless of the enemy's attacks. Hence, we present On-line Estimators for Ad-hoc Task Allocation with Adversary (OEATA-A), a novel algorithm to have better estimations of the teammates' future behaviour, which includes identifying enemies among friends.

show abstract

“…Afterwards, we manipulate the victim's observations so that it will imitate the behavior induced by the deceptive policy, which leads the victim astray from the right trajectories. Different from maximizing the least preferred action's Q-values (Lin et al, 2020), our method generates adversarial perturbations by minimizing the KL-divergence between the deceptive policy and the victim policy using Projected Gradient Descent (PGD) (Madry et al, 2017).…”

Section: Introductionmentioning

confidence: 99%

Understanding Adversarial Attacks on Observations in Deep Reinforcement Learning

Qiaoben

Cui

Zhou³

et al. 2021

Preprint

View full text Add to dashboard Cite

Recent works demonstrate that deep reinforcement learning (DRL) models are vulnerable to adversarial attacks which can decrease the victim's total reward by manipulating the observations. Compared with adversarial attacks in supervised learning, it is much more challenging to deceive a DRL model since the adversary has to infer the environmental dynamics. To address this issue, we reformulate the problem of adversarial attacks in function space and separate the previous gradient based attacks into several subspace. Following the analysis of the function space, we design a generic two-stage framework in the subspace where the adversary lures the agent to a target trajectory or a deceptive policy. In the first stage, we train a deceptive policy by hacking the environment, and discover a set of trajectories routing to the lowest reward. The adversary then misleads the victim to imitate the deceptive policy by perturbing the observations. Our method provides a tighter theoretical upper bound for the attacked agent's performance than the existing approaches. Extensive experiments demonstrate the superiority of our method and we achieve the state-of-the-art performance on both Atari and MuJoCo environments.Preprint. Under review.

show abstract

On the Robustness of Cooperative Multi-Agent Reinforcement Learning

Cited by 44 publications

References 19 publications

Evaluating Robustness of Cooperative MARL: A Model-based Approach

Evaluating Robustness of Cooperative MARL: A Model-based Approach

Task-Based Ad-hoc Teamwork with Adversary

Understanding Adversarial Attacks on Observations in Deep Reinforcement Learning

Contact Info

Product

Resources

About