Constraining the Attack Space of Machine Learning Models with Distribution Clamping Preprocessing

Abstract

Preprocessing and outlier detection techniques have both been applied to neural networks to increase robustness with varying degrees of success. In this paper, we formalize the ideal preprocessor function as one that would take any input and set it to the nearest in-distribution input. In other words, we detect any anomalous pixels and set them such that the new input is in-distribution. We then illustrate a relaxed solution to this problem in the context of patch attacks. Specifically, we demonstrate that we can model constraints on the patch attack that specify regions as out of distribution. With these constraints, we are able to preprocess inputs successfully, increasing robustness on CARLA object detection.

An AI blue team playbook

“…One category of input-based defenses is digital defenses, which use various preprocessing functions to protect against evasion attacks that either bypass multi-sensor physical defenses or are executed through a public-facing API. These defenses can fall under the following categories: detect and remove the attack vector; [28][29][30] implement non-differentiable functions to obscure gradients; 31,32 sanitize the attack vector to eliminate adversarial perturbations; [33][34][35] and apply formal verification [36][37][38] or certification techniques [39][40][41] to provide performance guarantees. Defenses applied to training data protect against poisoning attacks by filtering out potentially poisoned data samples [42][43][44][45][46] .…”

“…Model input Adversarial detection and removal, [28][29][30] input sanitization, [33][34][35] gradient obfuscation, 31,32 provable defenses [36][37][38][39][40][41] Remove adversarial perturbation, increase attack cost, or provide performance guarantees…”

“…There are numerous detection and removal defenses available. 28,29 Implementing and testing them can aid in identifying the specific algorithm that is most effective in meeting the system requirements and defending against the threat model. Generally, simpler defense algorithms execute more quickly, leading to reduced latency and increased throughput.…”