2017
DOI: 10.1167/17.10.1147
|View full text |Cite
|
Sign up to set email alerts
|

DeepGaze II: Predicting fixations from deep features over time and tasks

Abstract: Here we present DeepGaze II, a model that predicts where people look in images. The model uses the features from the VGG-19 deep neural network trained to identify objects in images. Contrary to other saliency models that use deep features, here we use the VGG features for saliency prediction with no additional fine-tuning (rather, a few readout layers are trained on top of the VGG features to predict saliency). The model is therefore a strong test of transfer learning. After conservative crossvalidation, Deep… Show more

Help me understand this report

Search citation statements

Order By: Relevance

Paper Sections

Select...
1
1
1

Citation Types

3
175
0

Year Published

2017
2017
2022
2022

Publication Types

Select...
5
2
1
1

Relationship

0
9

Authors

Journals

citations
Cited by 135 publications
(178 citation statements)
references
References 11 publications
3
175
0
Order By: Relevance
“…The model used Krizhevsky network to compute filter responses and a full convolution to learn the saliency model. Further more, a probabilistic model is introduced [97], which used VGG-19 features, incorporated center bias, and used a maximum likelihood learning to train the model. 8) Other models: Several other saliency models do not fit to the previously mentioned categories.…”
Section: ) Bayesian Modelsmentioning
confidence: 99%
“…The model used Krizhevsky network to compute filter responses and a full convolution to learn the saliency model. Further more, a probabilistic model is introduced [97], which used VGG-19 features, incorporated center bias, and used a maximum likelihood learning to train the model. 8) Other models: Several other saliency models do not fit to the previously mentioned categories.…”
Section: ) Bayesian Modelsmentioning
confidence: 99%
“…More recently, a plethora of deep learning based methods have been proposed for static saliency prediction. Kümmerer et al [26,27] proposed two deep saliency prediction networks, DeepGaze I and DeepGaze II, that was built on the AlexNet [25] and VGG-19 [40] models respectively. Pan et al [33] used a GAN to generate saliency maps.…”
Section: Related Workmentioning
confidence: 99%
“…We tested how well correlated the model's success rates (averaged over n=100 repetitions) were with human success rates (averaged over n=39 participants) across all images. For this analysis, we employed a saliency map generated by the DeepGaze algorithm (Kümmerer, Wallis, and Bethge 2016), rather than the frequency-tuned salient region detection algorithm (Methods). Remarkably, the model's performance strongly correlated with human performance, across images (Fig.…”
Section: Model Performance Mimics Human Performance Quantitativelymentioning
confidence: 99%