Models and Framework for Adversarial Attacks on Complex Adaptive Systems

Abstract

We introduce the paradigm of adversarial attacks that target the dynamics of Complex Adaptive Systems (CAS).To facilitate the analysis of such attacks, we present multiple approaches to the modeling of CAS as dynamical, datadriven, and game-theoretic systems, and develop quantitative definitions of attack, vulnerability, and resilience in the context of CAS security. Furthermore, we propose a comprehensive set of schemes for classification of attacks and attack surfaces in CAS, complemented with examples of practical attacks. Building on this foundation, we propose a framework based on reinforcement learning for simulation and analysis of attacks on CAS, and demonstrate its performance through three real-world case studies of targeting power grids, destabilization of terrorist organizations, and manipulation of machine learning agents. We also discuss potential mitigation techniques, and remark on future research directions in analysis and design of secure complex adaptive systems.

Security in the Internet of Things Application Layer: Requirements, Threats, and Solutions

“…Hence, there is a wide range of attacks against the involved devices that can destroy confidentiality. These attacks are man-in-the-middle attacks, eavesdropping attacks, model identification attacks against the used machine learning techniques, and parameter inference attacks against controllers [104]. Moreover, in ITSs, it is crucially important to ensure data integrity regarding communication and computation between different system devices, such as vehicles, traffic controllers, and roadside infrastructures.…”

Multiparty Dynamics and Failure Modes for Machine Learning and Artificial Intelligence

“…This definition does not require the failure to be due to malicious behavior on the part of any agent, nor does it forbid it. Note also that the definition does not require failure of the system, as in Behzadan and Munir's categorization of adversarial attacks [25], nor does it make any assumptions about type of the agents, such as the type of learning or optimization system used. (The multi-agent cases implicitly preclude agents from being either strongly boxed, as Drexler proposed [26], or oracular, as discussed by Armstrong [27].…”