Top-down saliency detection driven by visual classification

Abstract

This paper presents an approach for saliency detection able to emulate the integration of the top-down (task-controlled) and bottom-up (sensory information) processes involved in human visual attention. In particular, we first learn how to generate saliency when a specific visual task has to be accomplished. Afterwards, we investigate if and to what extent the learned saliency maps can support visual classification in nontrivial cases. To achieve this, we propose SalClass-Net, a CNN framework consisting of two networks jointly trained: a) the first one computing top-down saliency maps from input images, and b) the second one exploiting the computed saliency maps for visual classification. To test our approach, we collected a dataset of eye-gaze maps, using a Tobii T60 eye tracker, by asking several subjects to look at images from the Stanford Dogs dataset, with the objective of distinguishing dog breeds.Performance analysis on our dataset and other saliency benchmarking datasets, such as POET, showed that Sal-ClassNet outperforms state-of-the-art saliency detectors, such as SalNet and SALICON. Finally, we also analyzed the performance of SalClassNet in a fine-grained recognition task and found out that it yields enhanced classification accuracy compared to Inception and VGG-19 classifiers. The achieved results, thus, demonstrate that 1) conditioning saliency detectors with object classes reaches state-of-the-art performance, and 2) explicitly providing top-down saliency maps to visual classifiers enhances accuracy.

Desmoking Laparoscopy Surgery Images Using an Image-to-Image Translation Guided by an Embedded Dark Channel

“…• In [33] is proposed a method based on a convolutional neural network oriented to classify tasks that receive two inputs: an RGB image and its corresponding salience map (S) (a salience map is an image that tries to predict human fixations). The obtained results of this method show that performance improves when meaningful data are incorporated.…”

A Saliency-Based Convolutional Neural Network for Table and Chart Detection in Digitized Documents

“…In theory, the saliency maps could be the only expected output of the model, and we could train it by just providing the correct output as supervision. However, [7] recently showed that posing additional constraints to saliency detection -for example, forcing the saliency maps to identify regions that are also class-discriminative -improves output accuracy. This is highly desirable in our case as output saliency maps may miss non-salient regions (e.g., regular text) inside salient regions (e.g., table borders), while it is preferable to obtain maps that entirely cover the objects of interest.…”

Lire

“…• We investigated and developed the new top-down saliency detection approach driven by visual classification, which showed promising performance on common saliency detection evaluation datasets [84].…”

“…We shared our experience regarding how multimedia researchers can apply their knowledge in the medical field and published the article in the ACM multimedia Brave New Idea track [116]. In addition to the DeepEIR system [25,26,61,62,63,64,80,87,90,91,92,93,94,95,96,97,98,99,100,101,102,115,117,118,119,120,121,129] and side applications of its subsystems [13,14,15,16,55,84,113,122], this can be seen as an important contribution of this thesis to the research society.…”