On the Robustness of Domain Constraints

Sheatsley, Ryan; Hoak, Blaine; Pauley, Eric; Beugin, Yohan; Weisman, Michael J.; McDaniel, Patrick

doi:10.48550/arxiv.2105.08619

Cited by 4 publications

(5 citation statements)

References 43 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Most of the constrained attacks in feature space are an upgrade of traditional computer vision attacks like JSMA [16], C&W [17] or FGSM [18]. The majority of these attacks are evaluated in Network Intrusion Detection Systems (NIDS) [6], [10], [13], [19], [20] and few other domains: Cyber-Physical Systems [7], Twitter bot detection and website fingerprinting [21], and credit scoring [3], [22], [23]. Every attack handles different types of constraints, resulting in loosely realistic to totally realistic adversarial examples.…”

Section: A Constrained Adversarial Attacks In Feature Spacementioning

confidence: 99%

“…Sheatsley et al [6] integrate constraint resolution into JSMA attack to limit feature values in the range dictated by their primary feature. Later on, for the same case study, Sheatsley et al [20], learned boolean constraints with Valiant algorithm and introduced Constrained Saliency Projection (CSP), an improved iterative version of their previous attack. The bottleneck of this attack is the process of learning the constraints.…”

Section: A Constrained Adversarial Attacks In Feature Spacementioning

confidence: 99%

See 1 more Smart Citation

On The Empirical Effectiveness of Unrealistic Adversarial Hardening Against Realistic Adversarial Attacks

Dyrmishi¹,

Ghamizi²,

Simonetto³

et al. 2022

Preprint

View full text Add to dashboard Cite

While the literature on security attacks and defense of Machine Learning (ML) systems mostly focuses on unrealistic adversarial examples, recent research has raised concern about the under-explored field of realistic adversarial attacks and their implications on the robustness of real-world systems. Our paper paves the way for a better understanding of adversarial robustness against realistic attacks and makes two major contributions. First, we conduct a study on three real-world use cases (text classification, botnet detection, malware detection)) and five datasets in order to evaluate whether unrealistic adversarial examples can be used to protect models against realistic examples. Our results reveal discrepancies across the use cases, where unrealistic examples can either be as effective as the realistic ones or may offer only limited improvement. Second, to explain these results, we analyze the latent representation of the adversarial examples generated with realistic and unrealistic attacks. We shed light on the patterns that discriminate which unrealistic examples can be used for effective hardening. We release our code, datasets and models to support future research in exploring how to reduce the gap between unrealistic and realistic adversarial attacks.

show abstract

Section: A Constrained Adversarial Attacks In Feature Spacementioning

confidence: 99%

Section: A Constrained Adversarial Attacks In Feature Spacementioning

confidence: 99%

On The Empirical Effectiveness of Unrealistic Adversarial Hardening Against Realistic Adversarial Attacks

Dyrmishi¹,

Ghamizi²,

Simonetto³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Initially introduced in vision [21], [22], evasion has now been demonstrated for a wide variety of domains including audio [30]- [32] and text [33]- [36]. Attacks can also be physically realized [31], [37], [38] and constrained such that samples generated for evasion preserve input semantics [32], [34], [39]- [42].…”

Section: Risksmentioning

confidence: 99%

SoK: Machine Learning Governance

Chandrasekaran,

Jia,

Thudi

et al. 2021

Preprint

View full text Add to dashboard Cite

The application of machine learning (ML) in computer systems introduces not only many benefits but also risks to society. In this paper, we develop the concept of ML governance to balance such benefits and risks, with the aim of achieving responsible applications of ML. Our approach first systematizes research towards ascertaining ownership of data and models, thus fostering a notion of identity specific to ML systems. Building on this foundation, we use identities to hold principals accountable for failures of ML systems through both attribution and auditing. To increase trust in ML systems, we then survey techniques for developing assurance, i.e., confidence that the system meets its security requirements and does not exhibit certain known failures. This leads us to highlight the need for techniques that allow a model owner to manage the life cycle of their system, e.g., to patch or retire their ML system. Put altogether, our systematization of knowledge standardizes the interactions between principals involved in the deployment of ML throughout its life cycle. We highlight opportunities for future work, e.g., to formalize the resulting game between ML principals.

show abstract

“…Our observation that a proactive defender can observe each step of the pipeline is similar in idea to statistical testing at each of the layers of a computer vision model [45,55,59]. Sheatsley et al [66] make similar observations, and propose a data-driven approach to learn constraints from data enabling robustness. Concurrent work by Hussain et al [38] consider ASR systems.…”

Section: Related Workmentioning

confidence: 99%

On the Exploitability of Audio Machine Learning Pipelines to Surreptitious Adversarial Examples

Travers,

Licollari,

Wang

et al. 2021

Preprint

View full text Add to dashboard Cite

Machine learning (ML) models are known to be vulnerable to adversarial examples. Applications of ML to voice biometrics authentication are no exception. Yet, the implications of audio adversarial examples on these real-world systems remain poorly understood given that most research targets limited defenders who can only listen to the audio samples. Conflating detectability of an attack with human perceptibility, research has focused on methods that aim to produce imperceptible adversarial examples which humans cannot distinguish from the corresponding benign samples. We argue that this perspective is coarse for two reasons: 1. Imperceptibility is impossible to verify; it would require an experimental process that encompasses variations in listener training, equipment, volume, ear sensitivity, types of background noise etc, and 2. It disregards pipeline-based detection clues that realistic defenders leverage. This results in adversarial examples that are ineffective in the presence of knowledgeable defenders. Thus, an adversary only needs an audio sample to be plausible to a human. We thus introduce surreptitious adversarial examples, a new class of attacks that evades both human and pipeline controls. In the white-box setting, we instantiate this class with a joint, multi-stage optimization attack. Using an Amazon Mechanical Turk user study, we show that this attack produces audio samples that are more surreptitious than previous attacks that aim solely for imperceptibility. Lastly we show that surreptitious adversarial examples are challenging to develop in the black-box setting.

show abstract

On the Robustness of Domain Constraints

Cited by 4 publications

References 43 publications

On The Empirical Effectiveness of Unrealistic Adversarial Hardening Against Realistic Adversarial Attacks

On The Empirical Effectiveness of Unrealistic Adversarial Hardening Against Realistic Adversarial Attacks

SoK: Machine Learning Governance

On the Exploitability of Audio Machine Learning Pipelines to Surreptitious Adversarial Examples

Contact Info

Product

Resources

About