Availability Attacks Create Shortcuts

Yu, Dahua; Zhang, Huishuai; Chen, Wei; Yin, Jian; Liu, Tie-Yan

doi:10.1145/3534678.3539241

Cited by 24 publications

(29 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This lets CUDA generation add higher amounts of noise in the image space, specifically along the edges in images, and makes it resilient to AT with small additive noise budgets. In Figure 2, with the help of t-SNE plots [51], we also find that the noises added by CUDA generation is not linearly separable while they are linearly separable for the existing works on unlearnability [54].…”

Section: Introductionmentioning

confidence: 93%

See 1 more Smart Citation

CUDA: Convolution-based Unlearnable Datasets

Sadasivan¹,

Soltanolkotabi²,

Feizi³

2023

Preprint

View full text Add to dashboard Cite

Large-scale training of modern deep learning models heavily relies on publicly available data on the web. This potentially unauthorized usage of online data leads to concerns regarding data privacy. Recent works aim to make unlearnable data for deep learning models by adding small, specially designed noises to tackle this issue. However, these methods are vulnerable to adversarial training (AT) and/or are computationally heavy. In this work, we propose a novel, model-free, Convolution-based Unlearnable DAtaset (CUDA) generation technique. CUDA is generated using controlled class-wise convolutions with filters that are randomly generated via a private key. CUDA encourages the network to learn the relation between filters and labels rather than informative features for classifying the clean data. We develop some theoretical analysis demonstrating that CUDA can successfully poison Gaussian mixture data by reducing the clean data performance of the optimal Bayes classifier. We also empirically demonstrate the effectiveness of CUDA with various datasets (CIFAR-10, CIFAR-100, ImageNet-100, and Tiny-ImageNet), and architectures DeIT, and MobileNetV2). Our experiments show that CUDA is robust to various data augmentations and training approaches such as smoothing, AT with different budgets, transfer learning, and fine-tuning. For instance, training a ResNet-18 on ImageNet-100 CUDA achieves only 8.96%, 40.08%, and 20.58% clean test accuracies with empirical risk minimization (ERM), L ∞ AT, and L 2 AT, respectively. Here, ERM on the clean training data achieves a clean test accuracy of 80.66%. CUDA exhibits unlearnability effect with ERM even when only a fraction of the training dataset is perturbed. Furthermore, we also show that CUDA is robust to adaptive defenses designed specifically to break it.

show abstract

Section: Introductionmentioning

confidence: 93%

“…Fowl et al [10] show that error-maximizing noises as well can make strong poison attacks. However, all these poisoning techniques do not offer data protection with AT [49,54]. Fu et al [11] proposes a min-min-max optimization technique to generate poisoned data that offers better unlearnability effects with AT.…”

Section: Related Workmentioning

confidence: 99%

CUDA: Convolution-based Unlearnable Datasets

Sadasivan¹,

Soltanolkotabi²,

Feizi³

2023

Preprint

View full text Add to dashboard Cite

show abstract

“…The training objective is designed to create a spurious correlation between the noisy image and the ground-truth labels. Concurrently, Yu et al [65] empirically investigate various types of availability attacks and show that almost all of them leverage these spurious features to create a shortcut within neural networks [17]. Yu et al [65] then propose a fast and scalable approach for perturbation generation by generating randomly-initialized linearly-separable perturbations which can generate availability attacks for an entire dataset in a few seconds.…”

Section: Related Workmentioning

confidence: 99%

“…Recently, there has been an increasing number of studies on hindering the unauthorized use of personal data for neural network image classifiers [13,28,66,15,16,65,56,45]. These methods tend to add an imperceptible amount of noise to the clean images so that while the data has the same appearance as the ground-truth, it cannot provide any meaningful patterns for the neural networks to learn.…”

Section: Introductionmentioning

confidence: 99%

“…These methods tend to add an imperceptible amount of noise to the clean images so that while the data has the same appearance as the ground-truth, it cannot provide any meaningful patterns for the neural networks to learn. As a result, such approaches, collectively known as availability attacks [3], claim that personal image data can be made unexploitable for the neural networks [28,65]. While there has been an abundance of research for designing better availability attacks, far too little attention has been paid to counter-attacks that might be employed by adversaries to break such precautionary measures.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

The Devil's Advocate: Shattering the Illusion of Unexploitable Data using Diffusion Models

Dolatabadi¹,

Erfani²

2023

Preprint

View full text Add to dashboard Cite

Protecting personal data against the exploitation of machine learning models is of paramount importance. Recently, availability attacks have shown great promise to provide an extra layer of protection against the unauthorized use of data to train neural networks. These methods aim to add imperceptible noise to clean data so that the neural networks cannot extract meaningful patterns from the protected data, claiming that they can make personal data "unexploitable." In this paper, we provide a strong countermeasure against such approaches, showing that unexploitable data might only be an illusion. In particular, we leverage the power of diffusion models and show that a carefully designed denoising process can defuse the ramifications of the data-protecting perturbations. We rigorously analyze our algorithm, and theoretically prove that the amount of required denoising is directly related to the magnitude of the data-protecting perturbations. Our approach, called AVATAR, delivers state-of-the-art performance against a suite of recent availability attacks in various scenarios, outperforming adversarial training. Our findings call for more research into making personal data unexploitable, showing that this goal is far from over.

show abstract

COLLIDER: A Robust Training Framework for Backdoor Data

Dolatabadi

Erfani

2023

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Deep neural networks (DNNs) are vulnerable to shortcut learning: rather than learning the intended task, they tend to draw inconclusive relationships between their inputs and outputs. Shortcut learning is ubiquitous among many failure cases of neural networks, and traces of this phenomenon can be seen in their generalizability issues, domain shift, adversarial vulnerability, and even bias towards majority groups. In this paper, we argue that this commonality in the cause of various DNN issues creates a significant opportunity that should be leveraged to find a unified solution for shortcut learning. To this end, we outline the recent advances in topological data analysis (TDA), and persistent homology (PH) in particular, to sketch a unified roadmap for detecting shortcuts in deep learning. We demonstrate our arguments by investigating the topological features of computational graphs in DNNs using two cases of unlearnable examples and bias in decision-making as our test studies. Our analysis of these two failure cases of DNNs reveals that finding a unified solution for shortcut learning in DNNs is not out of reach, and TDA can play a significant role in forming such a framework.

show abstract

Availability Attacks Create Shortcuts

Cited by 24 publications

References 13 publications

CUDA: Convolution-based Unlearnable Datasets

CUDA: Convolution-based Unlearnable Datasets

The Devil's Advocate: Shattering the Illusion of Unexploitable Data using Diffusion Models

COLLIDER: A Robust Training Framework for Backdoor Data

Contact Info

Product

Resources

About