Stealthy and Efficient Adversarial Attacks against Deep Reinforcement Learning

Sun, Jianwen; Zhang, Tianwei; Xie, Xiaofei; Ма, Лей; Yan, Zheng; Chen, Kangjie; Liu, Yang

doi:10.1609/aaai.v34i04.6047

Cited by 77 publications

(39 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In addition, the effectiveness of an universal adversarial attack against DRL interpretations (i.e., UADRLI) has been verified by the theoretical analysis [204], from which the attacker can add the crafted universal perturbation uniformly to the environment states in a maximum number of steps to incur minimal damage. In order to stealthily attack the DRL agents, the work in [205] has injected adversarial samples in a minimal set of critical moments while causing the most severe damage to the agent. Another work in [206] has formulated an optimization framework in a stealthy manner for finding an optimal attack for different measures of attack cost, and solve it with an offline or online setting.…”

Section: • Model Inversion By Casting the Model Inversion Taskmentioning

confidence: 99%

Trusted AI in Multi-agent Systems: An Overview of Privacy and Security for Distributed Learning

Ma¹,

Li²,

Kang³

et al. 2022

Preprint

View full text Add to dashboard Cite

Motivated by the advancing computational capacity of distributed end-user equipments (UEs), as well as the increasing concerns about sharing private data, there has been considerable recent interest in machine learning (ML) and artificial intelligence (AI) that can be processed on on distributed UEs. Specifically, in this paradigm, parts of an ML process are outsourced to multiple distributed UEs, and then the processed ML information is aggregated on a certain level at a central server, which turns a centralized ML process into a distributed one, and brings about significant benefits. However, this new distributed ML paradigm raises new risks of privacy and security issues. In this paper, we provide a survey of the emerging security and privacy risks of distributed ML from a unique perspective of information exchange levels, which are defined according to the key steps of an ML process, i.e.: i) the level of preprocessed data, ii) the level of learning models, iii) the level of extracted knowledge and, iv) the level of intermediate results. We explore and analyze the potential of threats for each information exchange level based on an overview of the current state-of-the-art attack mechanisms, and then discuss the possible defense methods against such threats. Finally, we complete the survey by providing an outlook on the challenges and possible directions for future research in this critical area.

show abstract

Section: • Model Inversion By Casting the Model Inversion Taskmentioning

confidence: 99%

Trusted AI in Multi-agent Systems: An Overview of Privacy and Security for Distributed Learning

Ma¹,

Li²,

Kang³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Different from adversarial attacks which usually act during the inference process of a neural model [17,38,49,53,63,63,66,74,84,85], backdoor attacks hack the model during training [10,22,40,51,61,62,75,82]. Defending against such attacks is challenging [8,23,37,41,57,73] because users have no idea of what kinds of poison has been injected into model training.…”

Section: Backdoor Attack and Defensementioning

confidence: 99%

Defending against Backdoor Attacks in Natural Language Generation

Fan¹,

Li²,

Meng³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…Similarly, Lin et al [16] proposed to attack during a chosen subset of time steps. Applications of this type of attacks to autonomous driving have been shown to be effective [17], [18]. Weng et al [19] showed that learning the dynamics of agents and environments improves the efficacy of the attack in comparison to model-free methods.…”

Section: Introductionmentioning

confidence: 99%

Targeted Attack on Deep RL-based Autonomous Driving with Learned Visual Patterns

Buddareddygari¹,

Zhang²,

Yang³

et al. 2021

Preprint

View full text Add to dashboard Cite

Recent studies demonstrated the vulnerability of control policies learned through deep reinforcement learning against adversarial attacks, raising concerns about the application of such models to risk-sensitive tasks such as autonomous driving. Threat models for these demonstrations are limited to (1) targeted attacks through real-time manipulation of the agent's observation, and (2) untargeted attacks through manipulation of the physical environment. The former assumes full access to the agent's states/observations at all times, while the latter has no control over attack outcomes. This paper investigates the feasibility of targeted attacks through visually learned patterns placed on physical object in the environment, a threat model that combines the practicality and effectiveness of the existing ones. Through analysis, we demonstrate that a pre-trained policy can be hijacked within a time window, e.g., performing an unintended self-parking, when an adversarial object is present. To enable the attack, we adopt an assumption that the dynamics of both the environment and the agent can be learned by the attacker. Lastly, we empirically show the effectiveness of the proposed attack on different driving scenarios, perform a location robustness test, and study the tradeoff between the attack strength and its effectiveness.

show abstract

Stealthy and Efficient Adversarial Attacks against Deep Reinforcement Learning

Cited by 77 publications

References 16 publications

Trusted AI in Multi-agent Systems: An Overview of Privacy and Security for Distributed Learning

Trusted AI in Multi-agent Systems: An Overview of Privacy and Security for Distributed Learning

Defending against Backdoor Attacks in Natural Language Generation

Targeted Attack on Deep RL-based Autonomous Driving with Learned Visual Patterns

Contact Info

Product

Resources

About