Adversarial Machine Learning Beyond the Image Domain

Zizzo, Giulio; Hankin, Chris; Maffeis, Sergio; Jones, Kevin

doi:10.1145/3316781.3323470

Cited by 32 publications

(16 citation statements)

References 7 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This has led to a sub-discipline of adversarial machine learning. In [17,18] we examine the vulnerability of such intrusion detection systems to adversarial attacks. The attacker is able to manipulate the data sent to an IDS and seeks to hide their presence.…”

Section: Discussionmentioning

confidence: 99%

Trustworthy Inter-connected Cyber-Physical Systems

Hankin

Barrère

2020

Critical Information Infrastructures Security

Self Cite

View full text Add to dashboard Cite

In this paper we identify some of the particular challenges that are encountered when trying to secure cyber-physical systems. We describe three of our current activities: the architecture of a system for monitoring cyber-physical systems; a new approach to modelling dependencies in such systems which leads to a measurement of the security of the system -interpreted as the least effort that an attacker has to expend to compromise the operation; and an approach to optimising the diversity of products used in a system with a view to slowing the propagation of malware. We conclude by discussing how these different threads of work contribute to meeting the challenges and identify possible avenues for future development, as well as providing some pointers to other work.

show abstract

Section: Discussionmentioning

confidence: 99%

Trustworthy Inter-connected Cyber-Physical Systems

Hankin

Barrère

2020

Critical Information Infrastructures Security

Self Cite

View full text Add to dashboard Cite

show abstract

“…This is known as the inverse featuremapping problem [12,32,58]. Many works on problem-space attacks have been explored on different domains: text [3,43], PDFs [22,41,45,46,74], Windows binaries [38,59,60], Android apps [23,31,75], NIDS [6,7,20,28], ICS [76], and Javascript source code [58]. However, each of these studies has been conducted empirically and followed some inferred best practices: while they share many commonalities, it has been unclear how to compare them and what are the most relevant characteristics that should be taken into account while designing such attacks.…”

Section: Related Workmentioning

confidence: 99%

“…valid, inconspicuous member of the considered domain, and robust to non-ML preprocessing. Existing work investigated problem-space attacks on text [3,43], malicious PDFs [12,22,41,45,46,74], Android malware [23,75], Windows malware [38,60], NIDS [6,7,20,28], ICS [76], source code attribution [58], malicious Javascript [27], and eyeglass frames [62]. However, while there is a good understanding on how to perform feature-space attacks [16], it is less clear what the requirements are for an attack in the problem space, and how to compare strengths and weaknesses of existing solutions in a principled way.…”

Section: Introductionmentioning

confidence: 99%

Intriguing Properties of Adversarial ML Attacks in the Problem Space

Pierazzi

Pendlebury

Cortellazzi

et al. 2020

2020 IEEE Symposium on Security and Privacy (SP)

191

178

View full text Add to dashboard Cite

Recent research efforts on adversarial ML have investigated problem-space attacks, focusing on the generation of real evasive objects in domains where, unlike images, there is no clear inverse mapping to the feature space (e.g., software). However, the design, comparison, and real-world implications of problem-space attacks remain underexplored.This paper makes two major contributions. First, we propose a novel formalization for adversarial ML evasion attacks in the problem-space, which includes the definition of a comprehensive set of constraints on available transformations, preserved semantics, robustness to preprocessing, and plausibility. We shed light on the relationship between feature space and problem space, and we introduce the concept of side-effect features as the byproduct of the inverse feature-mapping problem. This enables us to define and prove necessary and sufficient conditions for the existence of problem-space attacks. We further demonstrate the expressive power of our formalization by using it to describe several attacks from related literature across different domains.Second, building on our formalization, we propose a novel problem-space attack on Android malware that overcomes past limitations. Experiments on a dataset with 170K Android apps from 2017 and 2018 show the practical feasibility of evading a state-of-the-art malware classifier along with its hardened version. Our results demonstrate that "adversarial-malware as a service" is a realistic threat, as we automatically generate thousands of realistic and inconspicuous adversarial applications at scale, where on average it takes only a few minutes to generate an adversarial app. Yet, out of the 1600+ papers on adversarial ML published in the past six years, roughly 40 focus on malware [15]-and many remain only in the feature space.Our formalization of problem-space attacks paves the way to more principled research in this domain. We responsibly release the code and dataset of our novel attack to other researchers, to encourage future work on defenses in the problem space.

show abstract

“…As ICSs are commonly attacked by state-sponsored actors, assuming the most knowledgeable adversary is not unrealistic. Related researches [1,4,32] use the white-box threat model as well. If the retraining schedule is not known, the attacker can still apply the same algorithms to calculate the poisoning samples, estimate the maximal retraining period, and increase the intervals between the poison injections to become larger than this estimation.…”

Section: Industrial Control Systemsmentioning

confidence: 99%

Poisoning attacks on cyber attack detectors for industrial control systems

Kravchik

Biggio

Shabtai

2021

Proceedings of the 36th Annual ACM Symposium on Applied Computing

View full text Add to dashboard Cite

Recently, neural network (NN)-based methods, including autoencoders, have been proposed for the detection of cyber attacks targeting industrial control systems (ICSs). Such detectors are often retrained, using data collected during system operation, to cope with the natural evolution (i.e., concept drift) of the monitored signals. However, by exploiting this mechanism, an attacker can fake the signals provided by corrupted sensors at training time and poison the learning process of the detector such that cyber attacks go undetected at test time. With this research, we are the first to demonstrate such poisoning attacks on ICS cyber attack online NN detectors. We propose two distinct attack algorithms, namely, interpolation-and back-gradient based poisoning, and demonstrate their effectiveness on both synthetic and real-world ICS data. We also discuss and analyze some potential mitigation strategies.

show abstract

Adversarial Machine Learning Beyond the Image Domain

Cited by 32 publications

References 7 publications

Trustworthy Inter-connected Cyber-Physical Systems

Trustworthy Inter-connected Cyber-Physical Systems

Intriguing Properties of Adversarial ML Attacks in the Problem Space

Poisoning attacks on cyber attack detectors for industrial control systems

Contact Info

Product

Resources

About