API-Net: Robust Generative Classifier via a Single Discriminator

Towards robust inference against distribution shifts in computer vision

“…It's a thorn in the side of the modern machine learning community that undermines the reliability of DNN models in various domains [112][113][114][115][116][117][118][119] and settings [120][121][122][123][124][125], causing serious security issues in computer vision systems, like face recognition [126] and autonomous vehicle [127]. Therefore, a line of related research [47,48,51,77,85,[128][129][130][131] has attracted significant attention in the machine learning community.…”

“…What's worse, the development of defenders [51,77] is relatively slower than the progress on attackers [120]. Generally, the defenders fall into the following five categories: adversarial training [47,48], data augmentation [132], generative classifier [128,129], de-noising [51,77], and certified defense [130]. Among them, some defenders [85,131] are inspired by biological vision systems.…”

“…To increase the adversarial robustness, a variety of methods are proposed as defenders. Generally, the defenders fall into the following five categories: 1) adversarial training [47,48] that intuitively generates adversarial examples to train the model, 2) data augmentation [132] that regards the adversarial vulnerability as the lack of training data, 3) generative classifier [128,129] that attempts to find which category better generate the input image rather than directly outputs the prediction based on likelihood, 4) de-noising methods [51,77] that consider adversarial perturbations as noises, which can be eliminated by either pre-network or in-network feature purification, and 5) certified defense [130], which tries to provide a safe bound of each image so any perturbation within the bound will not change the prediction.…”

“…Despite of various defenders proposed to improve the adversarial robustness, a universal remedy that can proactively defend against all the known and unknown attackers is still absent. Generally, the existing defenders fall into the following four categories: adversarial training [47,48,129], data augmentation [132], denoising [51,77], and certified defense [130]. In Section 6.5 we will systematically revisit them and compare them to the proposed CiiV from a causal viewpoint.…”

“…Specifically, adversarial training and its variants [47,48,129] together with some certified defenders like randomized smoothing [130] design some additive noises ε to imitate adversarial perturbations, then undermine the confounding effect by asking the model to be robust against ε. On the other hand, the de-noising approaches, no matter the pre-network de-noising [49][50][51] or the in-network de-noising [77,242] consider confounders to be explicitly removable patterns.…”