Do Saliency Models Detect Odd-One-Out Targets? New Datasets and Evaluations

Abstract

Recent advances in the field of saliency have concentrated on fixation prediction, with benchmarks reaching saturation. However, there is an extensive body of works in psychology and neuroscience that describe aspects of human visual attention that might not be adequately captured by current approaches. Here, we investigate singleton detection, which can be thought of as a canonical example of salience. We introduce two novel datasets, one with psychophysical patterns and one with natural odd-one-out stimuli. Using these datasets we demonstrate through extensive experimentation that nearly all saliency algorithms do not adequately respond to singleton targets in synthetic and natural images. Furthermore, we investigate the effect of training state-of-the-art CNN-based saliency models on these types of stimuli and conclude that the additional training data does not lead to a significant improvement of their ability to find odd-one-out targets.

DeepRare: Generic Unsupervised Visual Attention Models

“…We use 4 datasets namely OSIE [30], MIT1003 [31], P 3 , and O 3 datasets [22] to validate our results. The OSIE dataset contains information at three levels: pixel-level image attributes, object-level attributes, and semantic-level attributes.…”

“…A second drawback of the DNN-based models is that in addition to not take into account low-level features surprise level, DNN-based models are not generic enough to adapt to new images which are different enough from the training dataset. Indeed, recently, [22] introduced two novel datasets, one based on psycho-physical patterns (P 3 ) and one based on natural odd-one-out (O 3 ) stimuli. They showed that while DNN-based models are good in MIT dataset on natural images, their results drastically drop on P 3 and O 3 .…”

“…Based on the new datasets in [22], DeepRare2019 [25] provides a new deep-feature saliency model by mixing deep features and the philosophy of an existing classical model [7]. This model is efficient on all the datasets, with no need for any training and efficient in terms of computation even on CPU.…”

“…In a section 2 DR21 is described and the threshold on feature rarity is used to show how the DNN features rarity can become explainable. In section 3 this model is tested on the datasets proposed in [22] but also on an additional dataset. We finally discuss and conclude on the pertinence of the come back of the feature engineering models.…”

“…We compare our model to other models on P 3 and O 3 datasets. According to [22], they observe that most classical models perform better on P 3 than DNN-based models. In contrast, DNN-based models perform better on O 3 .…”