Natural Adversarial Objects

Abstract

Although state-of-the-art object detection methods have shown compelling performance, models often are not robust to adversarial attacks and out-of-distribution data. We introduce a new dataset, Natural Adversarial Objects (NAO), to evaluate the robustness of object detection models. NAO contains 7,934 images and 9,943 objects that are unmodified and representative of real-world scenarios, but cause state-of-the-art detection models to misclassify with high confidence. The mean average precision (mAP) of EfficientDet-D7 drops 74.5% when evaluated on NAO compared to the standard MSCOCO validation set.Moreover, by comparing a variety of object detection architectures, we find that better performance on MSCOCO validation set does not necessarily translate to better performance on NAO, suggesting that robustness cannot be simply achieved by training a more accurate model.We further investigate why examples in NAO are difficult to detect and classify. Experiments of shuffling image patches reveal that models are overly sensitive to local texture. Additionally, using integrated gradients and background replacement, we find that the detection model is reliant on pixel information within the bounding box, and insensitive to the background context when predicting class labels. NAO can be downloaded here.

Trapped in Texture Bias? A Large Scale Comparison of Deep Instance Segmentation

“…Both leverage style transfer to simulate novel object appearances. The closest real-life alternative is the Natural Adversarial Objects (NAO) dataset [39]. It depicts a more realistic out-of-distribution setting but does not allow to control for pose and perspective, i.e.…”

“…Despite their remarkable success in computer vision, deep neural networks still struggle in many challenging real-world scenarios [70,47,52,45]. One specific example are naturally adversarial objects [39]. Consider for instance a pedestrian with an unconventionally textured dress or a rare horse statue made out of bronze.…”

“…The model might have seen many pedestrians or natural horses during training but still fails to detect these rare or unseen examples, often with high confidence. Generalizing to such instances is typically described as out-of-distribution robustness [28,39]. For classification, [28] suggest that improvements in this direction are more likely to come from computer vision architectures than from existing data augmentation or additional public datasets.…”