Sample Complexity Bounds for Robustly Learning Decision Lists against Evasion Attacks

Abstract

Membership inference attacks (MIAs) aim to determine whether a specific sample was used to train a predictive model. Knowing this may indeed lead to a privacy breach. Most MIAs, however, make use of the model's prediction scores - the probability of each output given some input - following the intuition that the trained model tends to behave differently on its training data. We argue that this is a fallacy for many modern deep network architectures. Consequently, MIAs will miserably fail since overconfidence leads to high false-positive rates not only on known domains but also on out-of-distribution data and implicitly acts as a defense against MIAs. Specifically, using generative adversarial networks, we are able to produce a potentially infinite number of samples falsely classified as part of the training data. In other words, the threat of MIAs is overestimated, and less information is leaked than previously assumed. Moreover, there is actually a trade-off between the overconfidence of models and their susceptibility to MIAs: the more classifiers know when they do not know, making low confidence predictions, the more they reveal the training data.

An Introduction to Adversarially Robust Deep Learning

“…Specifically, determining the conditions under which robust learning is possible, and to what extent, is one of the major problems in the field. In this regard, [128] have made several important contributions. They showed that robust learning is impossible in the distribution-free setting even when the adversary is restricted to perturbing just a single bit of the input.…”

When are Local Queries Useful for Robust Learning?

“…In this section, we show that the amount of data needed to ρ-robustly learn conjunctions under the uniform distribution has an exponential dependence on the adversary's budget ρ when the learner only has access to the EX and LMQ oracles. Here, the lower bound on the sample drawn from the example oracle is 2 ρ , which is the same as the lower bound for monotone conjunctions derived in Gourdeau et al (2022), and the local membership query lower bound is 2 ρ−1 . The result relies on showing there there exists a family of conjunctions that remain indistinguishable from each other on any sample of size 2 ρ and any sequence of 2 ρ−1 LMQs with constant probability.…”

“…We use the term robustness threshold from Gourdeau et al (2021) to denote an adversarial budget function ρ : N → R of the input dimension n such that, if the adversary is allowed perturbations of magnitude ρ(n), then there exists a sample-efficient ρ(n)-robust learning algorithm, and if the adversary's budget is ω(ρ(n)), then there does not exist such an algorithm. Robustness thresholds are distribution-dependent when the learner only has access to the example oracle EX, as seen in (Gourdeau et al, 2021(Gourdeau et al, , 2022. Now, since the local membership query lower bound above has an exponential dependence on ρ, any perturbation budget ω(log n) will require a sample and query complexity that is superpolynomial in n, giving the following corollary.…”

“…They also show that monotone conjunctions have a robustness threshold of Θ(log n) under log-Lipschitz distributions, meaning that this class is efficiently robustly learnable against an adversary that can perturb log n bits of the input, but if an adversary is allowed to perturb ρ = ω(log n) bits of the input, there does not exist a sample-efficient learning algorithm for this problem. Gourdeau et al (2021) extended this result to the class of monotone decision lists and Gourdeau et al (2022) showed a sample complexity lower bound for monotone conjunctions that is exponential in ρ and that the robustness threshold of decision lists is also Θ(log n). Finally, Diakonikolas et al (2020) and Bhattacharjee et al (2021) have used online learning algorithms for robust learning with respect to the constant-in-the-ball notion of robustness.…”

When to Trust AI: Advances and Challenges for Certification of Neural Networks

“…A natural question then arises as to whether one can learn a model that is guaranteed to be robust. Building on the positive and negative theoretical results in the case of robust learning against evasion attacks [58], [59], [60], [61], it would be interesting to generalise these results to neural network models and development of implementable frameworks that can provide provable guarantees on robustness.…”