Detecting Backdoor Poisoning Attacks on Deep Neural Networks by Heatmap Clustering

Abstract

Predicitions made by neural networks can be fraudulently altered by so-called poisoning attacks. A special case are backdoor poisoning attacks. We study suitable detection methods and introduce a new method called Heatmap Clustering. There, we apply a k-means clustering algorithm on heatmaps produced by the state-of-the-art explainable AI method Layer-wise relevance propagation. The goal is to separate poisoned from un-poisoned data in the dataset. We compare this method with a similar method, called Activation Clustering, which also uses k-means clustering but applies it on the activation of certain hidden layers of the neural network as input. We test the performance of both approaches for standard backdoor poisoning attacks, label-consistent poisoning attacks and label-consistent poisoning attacks with reduced amplitude stickers. We show that Heatmap Clustering consistently performs better than Activation Clustering. However, when considering labelconsistent poisoning attacks, the latter method also yields good detection performance.

Data Isotopes for Data Provenance in DNNs

“…We draw inspiration from defenses against poisoning, backdoor, and membership inference attacks, which are all related to isotopes (see §2), to identify techniques that could detect or disrupt isotopes. For example, A could try to detect isotopes using existing methods for spurious correlation detection [48,64] or by analyzing F to detect isotope-induced changes [10,25,51,59,69,71]. To disrupt isotopes, A could use adversarial augmentations during training [54,62], modify F 's outputs to harm V's performance [32,63], or selectively retrain F so it forgets isotope features [40].…”

“…Since marks increase the probability of the marked label 𝑦 𝑗 for marked inputs, the feature-space region associated with 𝑦 𝑗 may exhibit isotope-specific behaviors. Several defenses against backdoor attacks use feature inspection to detect backdoors [10,25,51,59,69,71].…”

SoK: How Robust is Image Classification Deep Neural Network Watermarking?

“…The model's training data is scraped from public sources and can contain a small number of poisoned images [6]. A dataset sanitation process aims to filter or augment poisoned images [8,26,67] before training the model using a robust training approach [4,31,43,80]. After training, the defender can choose to inspect or repair the model using a limited amount of trustworthy data [44,46,79] before deployment when they suspect backdooring.…”