Minimalistic Attacks: How Little It Takes to Fool Deep Reinforcement Learning Policies

Abstract

Recent studies have revealed that neural network based policies can be easily fooled by adversarial examples. However, while most prior works analyze the effects of perturbing every pixel of every frame assuming white-box policy access, in this paper we take a more restrictive view towards adversary generation -with the goal of unveiling the limits of a model's vulnerability. In particular, we explore minimalistic attacks by defining three key settings: (1) black-box policy access: where the attacker only has access to the input (state) and output (action probability) of an RL policy; (2) fractional-state adversary: where only several pixels are perturbed, with the extreme case being a single-pixel adversary; and (3) tactically-chanced attack: where only significant frames are tactically chosen to be attacked. We formulate the adversarial attack by accommodating the three key settings, and explore their potency on six Atari games by examining four fully trained state-of-the-art policies. In Breakout, for example, we surprisingly find that: (i) all policies showcase significant performance degradation by merely modifying 0.01% of the input state, and (ii) the policy trained by DQN is totally deceived by perturbation to only 1% frames.

S$$^{2}$$ES: a stationary and scalable knowledge transfer approach for multiagent reinforcement learning

“…Shannon entropy is another metric that can describe the action quality in the aspect of confidence in the decisionmaking process. Thus, we try to adopt the normalized Shannon entropy in [28] to the S 2 ES framework, named as S 2 ES-H. The normalized Shannon entropy can be written as…”

Adversarial robustness of deep reinforcement learning

“…does not comprehensively account for the scope of exploiting frame correlations. For this reason, even though [70] marks an advance over prior forms of adversarial attacks in RL, its adversary generation process may still be insufficient for unveiling the realizability of real-time attacks across many time-sensitive RL applications.…”

“…In Chapter 3, a minimalistic attack with the afore-stated strong assumptions being relaxed [70] has been investigated. In particular, a genetic algorithm (GA) [37] is used for the generation of selective perturbations (on selective pixels and frames), treating the policy as a black-box.…”

“…as function approximators which is trained in an end-to-end style [56]. Although DRL has shown remarkable performance on various complex tasks (e.g., board games [79], video games [56] and robotics control [46]), many studies on adversarial attacks [39,47,71,94] indicate that DRL policies can be easily deceived by the perturbations (e.g., human imperceptible pixel noises and inaccurate sensor measurement) on the observations of states.…”

“…The advancements in deep reinforcement learning (DRL 1 ) have demonstrated that deep neural networks (DNNs) as powerful function approximators can be trained to prescribe near-optimal actions on many complex tasks (e.g., Atari games [56], robotics control [46] and motor control [87]). Although remarkable achievements have been documented, many studies on adversarial attacks [39,47,71,94] have shown that DRL policies are vulnerable to adversarial perturbations in state observations. This inspires the studies on both the evaluation and improvement of DRL's adversarial robustness.…”