The Geometry of Adversarial Training in Binary Classification

Abstract

We establish an equivalence between a family of adversarial training problems for non-parametric binary classification and a family of regularized risk minimization problems where the regularizer is a nonlocal perimeter functional. The resulting regularized risk minimization problems admit exact convex relaxations of the type L 1 + (nonlocal) TV, a form frequently studied in image analysis and graph-based learning. A rich geometric structure is revealed by this reformulation which in turn allows us to establish a series of properties of optimal solutions of the original problem, including the existence of minimal and maximal solutions (interpreted in a suitable sense), and the existence of regular solutions (also interpreted in a suitable sense). In addition, we highlight how the connection between adversarial training and perimeter minimization problems provides a novel, directly interpretable, statistical motivation for a family of regularized risk minimization problems involving perimeter/total variation. The majority of our theoretical results are independent of the distance used to define adversarial attacks.

Eikonal depth: an optimal control approach to statistical depths

“…But on the other hand they may move many points short distances, which makes these adversaries in another sense stronger than the one associated with the breakdown point. Utilizing this type of adversary has been incredibly effective in improving generalization for statistical algorithms in the context of deep learning [25], and has sparked significant algorithmic and theoretical work [4,22,34,38,43].…”

Towards Consistency in Adversarial Classification

“…Another line of work [Pinot et al, 2020, Meunier et al, 2021, Pydi and Jog, 2021 have focused on a game theoretic approach for analyzing the adversarial risk having interest in the nature of equilibria between the classifier and the attacker. Recently, some researchers [Awasthi et al, 2021b, Bungert et al, 2021 proved encouraging results on the existence of an optimal Bayes classifier in the adversarial setting under mild assumptions.…”