Blackbox Attacks on Reinforcement Learning Agents Using Approximated Temporal Information

Zhao, Yiren; Shumailov, Ilia; Cui, Han; Gao, Xitong; Mullins, Robert; Anderson, Ross

doi:10.1109/dsn-w50199.2020.00013

Cited by 21 publications

(15 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Following this, a few works have further contributed to this rising topic of adversarial attacks in deep RL. Zhao et al [16] propose an approach to generate adversarial observations without previous knowledge on the network architecture and RL algorithm of the deep reinforcement learning agent. Instead of adding perturbations to the observations, Gleave et al [17] propose to train the agent with an adversarial policy.…”

Section: Related Workmentioning

confidence: 99%

“…The combined trajectory data can then be used to update the policy of the RL system. As deep RL systems typically inherit the vulnerability of deep neural networks to adversarial examples [15], some previous works have investigated the topic of adversarial attacks in deep RL [16,17]. Our work however is the first that deploys adversarial data augmentation in RL systems to improve their generalization capacity.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Generalization of Reinforcement Learning with Policy-Aware Adversarial Data Augmentation

Zhang¹,

Guo²

2021

Preprint

View full text Add to dashboard Cite

The generalization gap in reinforcement learning (RL) has been a significant obstacle that prevents the RL agent from learning general skills and adapting to varying environments. Increasing the generalization capacity of the RL systems can significantly improve their performance on real-world working environments. In this work, we propose a novel policy-aware adversarial data augmentation method to augment the standard policy learning method with automatically generated trajectory data. Different from the commonly used observation transformation based data augmentations, our proposed method adversarially generates new trajectory data based on the policy gradient objective and aims to more effectively increase the RL agent's generalization ability with the policy-aware data augmentation. Moreover, we further deploy a mixup step to integrate the original and generated data to enhance the generalization capacity while mitigating the over-deviation of the adversarial data. We conduct experiments on a number of RL tasks to investigate the generalization performance of the proposed method by comparing it with the standard baselines and the state-of-the-art mixreg approach. The results show our method can generalize well with limited training diversity, and achieve the state-of-the-art generalization test performance.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Generalization of Reinforcement Learning with Policy-Aware Adversarial Data Augmentation

Zhang¹,

Guo²

2021

Preprint

View full text Add to dashboard Cite

show abstract

“…These methods require both reverting the environment to a previous state and making unlimited queries to π v , which is not possible in real-time. Other black-box methods [10,29] use proxy models to approximate π v . Adversarial perturbations are generated from these proxy models.…”

Section: Related Workmentioning

confidence: 99%

Real-time Adversarial Perturbations against Deep Reinforcement Learning Policies: Attacks and Defenses

Tekgul¹,

Wang²,

Marchal³

et al. 2021

Preprint

View full text Add to dashboard Cite

Recent work has discovered that deep reinforcement learning (DRL) policies are vulnerable to adversarial examples. These attacks mislead the policy of DRL agents by perturbing the state of the environment observed by agents. They are feasible in principle but too slow to fool DRL policies in real time. We propose a new attack to fool DRL policies that is both effective and efficient enough to be mounted in real time. We utilize the Universal Adversarial Perturbation (UAP) method to compute effective perturbations independent of the individual inputs to which they are applied. Via an extensive evaluation using Atari 2600 games, we show that our technique is effective, as it fully degrades the performance of both deterministic and stochastic policies (up to 100%, even when the l ∞ bound on the perturbation is as small as 0.005). We also show that our attack is efficient, incurring an online computational cost of 0.027ms on average. It is faster compared to the response time (0.6ms on average) of agents with different DRL policies, and considerably faster than prior attacks (2.7ms on average). Furthermore, we demonstrate that known defenses are ineffective against universal perturbations. We propose an effective detection technique which can form the basis for robust defenses against attacks based on universal perturbations.

show abstract

“…Prior work has demonstrated that RL is vulnerable to input perturbation attacks, where the attacker modifies the agent's observation with the goal of degrading its performance; for example, using the FGSM [13] to attack three RL networks [5]. Other attacks reduce the number of adversarial examples needed to decrease the agent's reward [17] or trigger misbehavior of the agent after a delay [18]. The learning process of DQN itself can be attacked [19], by constructing a replica model of the victim and transferring adversarial examples from the FGSM and JSMA [16] techniques to craft adversarial examples.…”

Section: Related Workmentioning

confidence: 99%

On the Robustness of Cooperative Multi-Agent Reinforcement Learning

Lin

Dzeparoska

Zhang

et al. 2020

2020 IEEE Security and Privacy Workshops (SPW)

View full text Add to dashboard Cite

In cooperative multi-agent reinforcement learning (c-MARL), agents learn to cooperatively take actions as a team to maximize a total team reward. We analyze the robustness of c-MARL to adversaries capable of attacking one of the agents on a team. Through the ability to manipulate this agent's observations, the adversary seeks to decrease the total team reward.Attacking c-MARL is challenging for three reasons: first, it is difficult to estimate team rewards or how they are impacted by an agent mispredicting; second, models are non-differentiable; and third, the feature space is low-dimensional. Thus, we introduce a novel attack. The attacker first trains a policy network with reinforcement learning to find a wrong action it should encourage the victim agent to take. Then, the adversary uses targeted adversarial examples to force the victim to take this action.Our results on the StartCraft II multi-agent benchmark demonstrate that c-MARL teams are highly vulnerable to perturbations applied to one of their agent's observations. By attacking a single agent, our attack method has highly negative impact on the overall team reward, reducing it from 20 to 9.4. This results in the team's winning rate to go down from 98.9% to 0%.

show abstract

Blackbox Attacks on Reinforcement Learning Agents Using Approximated Temporal Information

Cited by 21 publications

References 12 publications

Generalization of Reinforcement Learning with Policy-Aware Adversarial Data Augmentation

Generalization of Reinforcement Learning with Policy-Aware Adversarial Data Augmentation

Real-time Adversarial Perturbations against Deep Reinforcement Learning Policies: Attacks and Defenses

On the Robustness of Cooperative Multi-Agent Reinforcement Learning

Contact Info

Product

Resources

About