Visual Attention: Deep Rare Features

Abstract

Human visual system is modeled in engineering field providing feature-engineered methods which detect contrasted/surprising/unusual data into images. This data is "interesting" for humans and leads to numerous applications. Deep learning (DNNs) drastically improved the algorithms efficiency on the main benchmark datasets. However, DNN-based models are counter-intuitive: surprising or unusual data is by definition difficult to learn because of its low occurrence probability. In reality, DNNs models mainly learn top-down features such as faces, text, people, or animals which usually attract human attention, but they have low efficiency in extracting surprising or unusual data in the images.In this paper, we propose a model called DeepRare2019 (DR) which uses the power of DNNs feature extraction and the genericity of feature-engineered algorithms. DR 1) does not need any training, 2) it takes less than a second per image on CPU only and 3) our tests on three very different eye-tracking datasets show that DR is generic and is always in the top-3 models on all datasets and metrics while no other model exhibits such a regularity and genericity. DeepRare2019 code can be found at https://github.com/numediart/VisualAttention-RareFamily.

Nighttime Driver Behavior Prediction Using Taillight Signal Recognition via CNN-SVM Classifier

“…Similar to [37], the fully connected and pooling layer from the VGG-19 network is removed. Then, in each layer and for each feature, the data rarity is calculated [38]. To be more precise, there are 2×64 features at level 1, and the data rarity is calculated for each.…”

DeepRare: Generic Unsupervised Visual Attention Models

“…However, they are not yet comparable to DNN-based models for general images datasets such as the MIT benchmark. Based on the new datasets in [29], DeepRare2019 [32] provides a new deep-feature saliency model by mixing deep features and the philosophy of an existing classical model [9]. This model is efficient on all the datasets, with no need for any additional training and efficient in terms of computation even on CPU.…”

“…In this article, we extend the approach in [32] where a framework called DeepRare is proposed which mixes the simplicity of the idea of rarity computation to find the most salient features with the advantages of deep features extraction. Indeed, rare features attract human attention as they are surprising compared to the other features within the image.…”

“…In addition to smoothing the data, we tested the fact of squaring the data after the smoothing. Tables 3 and 4 show the results for the chosen configuration in Section 2.5, which is the mix of thresholds 0 and 0.9 (1) not filtered, (2) using the filtering technique from [32], and (3) squared after the filtering technique. We can see that, in all cases, the filter followed by the square provides the best results.…”

“…We can see that, in all cases, the filter followed by the square provides the best results. When trying to cube the image or even more, results are worse, so we decided to keep as the final post processing scheme the filtering from [32] followed by the squared map. To summarize the difference between the DR19 and DR21 methods, Table 5 shows the main differences.…”