A single gradient step finds adversarial examples on random two-layers neural networks

Abstract

Daniely and Schacham recently showed that gradient descent finds adversarial examples on random undercomplete two-layers ReLU neural networks. The term "undercomplete" refers to the fact that their proof only holds when the number of neurons is a vanishing fraction of the ambient dimension. We extend their result to the overcomplete case, where the number of neurons is larger than the dimension (yet also subexponential in the dimension). In fact we prove that a single step of gradient descent suffices. We also show this result for any subexponential width random neural network with smooth activation function.

Adversarial Robustness is at Odds with Lazy Training

“…Inspired by the work of Shamir et al [2019], Daniely and Schacham [2020] prove that small 2 norm adversarial perturbations can be found by multi-step gradient descent method for random ReLU networks with small width -each layer has vanishing width compared to the previous layer. Bubeck et al [2021] generalize this result to two-layer randomly initialized networks with relatively large network width and show that a single step of gradient descent suffices to find adversarial examples. Bartlett et al [2021] further generalize this result to random multilayer ReLU networks.…”

“…Interestingly, linearity has been hypothesized as key reason for existence of adversarial examples in neural networks Goodfellow et al [2014]. Furthermore, this was used to prove existence of adversarial attacks for random neural networks Bubeck et al [2021], Bartlett et al [2021] and also provide the basis of our analysis in this work.…”

“…m = o(d)). More recently, work of Bubeck et al [2021] shows that a single gradient descent update finds adversarial examples for sufficiently wide randomly initialized ReLU networks. This was later extended to multi-layer networks with the work of Bartlett et al [2021].…”

“…We investigate the robustness of over-parametrized neural networks against inference time attack in the lazy training regime. Building on the ideas of Bubeck et al [2021], we show that such trained neural networks still suffer from adversarial examples, which can be found using a single gradient step.…”

“…

Recent works show that random neural networks are vulnerable against adversarial attacks [Daniely and Schacham, 2020] and that such attacks can be easily found using a single step of gradient descent [Bubeck et al, 2021]. In this work, we take it one step further and show that a single gradient step can find adversarial examples for networks trained in the so-called lazy regime.

…”