Towards Frequency-Based Explanation for Robust CNN

Wang, Zifan; Yang, Y.; Shrivastava, Ankit; Rawal, Varun; Ding, Zegang

doi:10.48550/arxiv.2005.03141

Cited by 10 publications

(8 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Instead the authors propose the texture hypothesis, stating that an inherent texturebias in the dataset can lead to a lack of robustness in CNNs. Similarly, other works [32] have reported a higher importance of texture-like high-frequency input features, which aligns with the vulnerability to high-frequency adversarial attacks [30]. Through expensive modification of the training dataset, exchanging the dataset's texture-bias to a shape-bias, the authors of [9] achieve improved classification robustness.…”

Section: Introductionmentioning

confidence: 56%

Revisiting the Shape-Bias of Deep Learning for Dermoscopic Skin Lesion Classification

Lucieri

Schmeisser

Balada

et al. 2022

Lecture Notes in Computer Science

View full text Add to dashboard Cite

It is generally believed that the human visual system is biased towards the recognition of shapes rather than textures. This assumption has led to a growing body of work aiming to align deep models' decision-making processes with the fundamental properties of human vision. The reliance on shape features is primarily expected to improve the robustness of these models under covariate shift. In this paper, we revisit the significance of shape-biases for the classification of skin lesion images. Our analysis shows that different skin lesion datasets exhibit varying biases towards individual image features. Interestingly, despite deep feature extractors being inclined towards learning entangled features for skin lesion classification, individual features can still be decoded from this entangled representation. This indicates that these features are still represented in the learnt embedding spaces of the models, but not used for classification. In addition, the spectral analysis of different datasets shows that in contrast to common visual recognition, dermoscopic skin lesion classification, by nature, is reliant on complex feature combinations beyond shape-bias. As a natural consequence, shifting away from the prevalent desire of shape-biasing models can even improve skin lesion classifiers in some cases.

show abstract

Section: Introductionmentioning

confidence: 56%

Revisiting the Shape-Bias of Deep Learning for Dermoscopic Skin Lesion Classification

Lucieri

Schmeisser

Balada

et al. 2022

Lecture Notes in Computer Science

View full text Add to dashboard Cite

show abstract

“…FcaNet [76] generalized the pre-processing of channel attention mechanism in the frequency domain. Wang et al, [77] applied the analysis of the connection between the distribution of frequency components in the input dataset to conduct the explanation of the CNN. Since the convolution operation in the spatial domain has been proven to be equivalent to the multiplication in the frequency domain [78], [79] performed video knowledge distillation in the frequency domain for action recognition.…”

Section: Frequency Domain Learningmentioning

confidence: 99%

TCGL: Temporal Contrastive Graph for Self-supervised Video Representation Learning

Liu¹,

Wang²,

Liu³

et al. 2021

Preprint

View full text Add to dashboard Cite

Video self-supervised learning is a challenging task, which requires significant expressive power from the model to leverage rich spatial-temporal knowledge and generate effective supervisory signals from large amounts of unlabeled videos. However, existing methods fail to increase the temporal diversity of unlabeled videos and ignore elaborately modeling multiscale temporal dependencies in an explicit way. To overcome these limitations, we take advantage of the multi-scale temporal dependencies within videos and proposes a novel video selfsupervised learning framework named Temporal Contrastive Graph Learning (TCGL), which jointly models the inter-snippet and intra-snippet temporal dependencies for temporal representation learning with a hybrid graph contrastive learning strategy. Specifically, a Spatial-Temporal Knowledge Discovering (STKD) module is first introduced to extract motion-enhanced spatialtemporal representations from videos based on the frequency domain analysis of discrete cosine transform. To explicitly model multi-scale temporal dependencies of unlabeled videos, our TCGL integrates the prior knowledge about the frame and snippet orders into graph structures, i.e., the intra-/inter-snippet Temporal Contrastive Graphs (TCG). Then, specific contrastive learning modules are designed to maximize the agreement between nodes in different graph views. To generate supervisory signals for unlabeled videos, we introduce an Adaptive Snippet Order Prediction (ASOP) module which leverages the relational knowledge among video snippets to learn the global context representation and recalibrate the channel-wise features adaptively. Experimental results demonstrate the superiority of our TCGL over the state-of-the-art methods on large-scale action recognition and video retrieval benchmarks. The code is publicly available at https://github.com/YangLiu9208/TCGL.

show abstract

“…Contrastive view from high-frequency component Next, we use the high-frequency component (HFC) of data as another additional contrastive view. The rationale arises from the facts that 1) learning over HFC of data is a main cause of achieving superior generalization ability [52] and 2) an adversary typically concentrates on HFC when manipulating an example to fool model's decision [53]. Let F and F −1 denote Fourier transformation and its inverse.…”

Section: Contrastive View From Adversarial Examplementioning

confidence: 99%

When Does Contrastive Learning Preserve Adversarial Robustness from Pretraining to Finetuning?

Fan¹,

Liu²,

Chen³

et al. 2021

Preprint

View full text Add to dashboard Cite

Contrastive learning (CL) can learn generalizable feature representations and achieve state-of-the-art performance of downstream tasks by finetuning a linear classifier on top of it. However, as adversarial robustness becomes vital in image classification, it remains unclear whether or not CL is able to preserve robustness to downstream tasks. The main challenge is that in the 'self-supervised pretraining + supervised finetuning' paradigm, adversarial robustness is easily forgotten due to a learning task mismatch from pretraining to finetuning. We call such challenge 'cross-task robustness transferability'. To address the above problem, in this paper we revisit and advance CL principles through the lens of robustness enhancement. We show that (1) the design of contrastive views matters: High-frequency components of images are beneficial to improving model robustness; (2) Augmenting CL with pseudo-supervision stimulus (e.g., resorting to feature clustering) helps preserve robustness without forgetting. Equipped with our new designs, we propose ADVCL, a novel adversarial contrastive pretraining framework. We show that ADVCL is able to enhance cross-task robustness transferability without loss of model accuracy and finetuning efficiency. With a thorough experimental study, we demonstrate that ADVCL outperforms the state-of-the-art self-supervised robust learning methods across multiple datasets (CIFAR-10, CIFAR-100 and STL-10) and finetuning schemes (linear evaluation and full model finetuning). Code is available at https://github.com/LijieFan/AdvCL.

show abstract

Towards Frequency-Based Explanation for Robust CNN

Cited by 10 publications

References 16 publications

Revisiting the Shape-Bias of Deep Learning for Dermoscopic Skin Lesion Classification

Revisiting the Shape-Bias of Deep Learning for Dermoscopic Skin Lesion Classification

TCGL: Temporal Contrastive Graph for Self-supervised Video Representation Learning

When Does Contrastive Learning Preserve Adversarial Robustness from Pretraining to Finetuning?

Contact Info

Product

Resources

About