Spatial Locality Weighting of Features Using Saliency Map with a Bag-of-Visual-Words Approach

Soares, Robson de Carvalho; Silva, Ilmério Reis da; Guliato, Denise

doi:10.1109/ictai.2012.151

Cited by 13 publications

(12 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Some works have investigated the usage of saliency as a mechanism to reduce the number of local features and reduce the computational complexity of SIFTbased BoW frameworks [1,23,25]. Other works, rather than completely discard the background information, have used saliency to weight the contribution of the foreground and the background simultaneously [9,41]. However, the usage of saliency models in the task of image retrieval has thus far been restricted to handcrafted CBIR features and handcrafted saliency models [14].…”

Section: Saliency Modelsmentioning

confidence: 99%

Saliency Weighted Convolutional Features for Instance Search

Mohedano

McGuinness

Giró-i-Nieto

et al. 2018

2018 International Conference on Content-Based Multimedia Indexing (CBMI)

View full text Add to dashboard Cite

This work explores attention models to weight the contribution of local convolutional representations for the instance search task. We present a retrieval framework based on bags of local convolutional features (BLCF) that benefits from saliency weighting to build an efficient image representation. The use of human visual attention models (saliency) allows significant improvements in retrieval performance without the need to conduct region analysis or spatial verification, and without requiring any feature fine tuning. We investigate the impact of different saliency models, finding that higher performance on saliency benchmarks does not necessarily equate to improved performance when used in instance search tasks. The proposed approach outperforms the state-of-the-art on the challenging INSTRE benchmark by a large margin, and provides similar performance on the Oxford and Paris benchmarks compared to more complex methods that use off-the-shelf representations.

show abstract

Section: Saliency Modelsmentioning

confidence: 99%

Saliency Weighted Convolutional Features for Instance Search

Mohedano

McGuinness

Giró-i-Nieto

et al. 2018

2018 International Conference on Content-Based Multimedia Indexing (CBMI)

View full text Add to dashboard Cite

show abstract

“…Following the second strategy, the works in [13,15] substitute these binary masks by a soft-pooling scheme over real-valued saliency maps. In particular, both works build over the BoW paradigm, and consider the continuous values of a saliency map to weigh the contribution of each visual word.…”

Section: Related Work In Saliency-based Object Recognitionmentioning

confidence: 99%

“…Models of visual attention, such as the one proposed by Itti et al [11] or Harel's graph implementation [12] are frequently used in literature for computing saliency maps. Various authors have shown how driving the processing to those particular areas with high values in the saliency maps improves the system performance in various computer vision tasks, such as image retrieval [13], object recognition [14,15], object tracking [16,17], or action recognition [18,19]. However, although much fundamental work has been done to generate good representations of visual saliency from still images or video content, their ap-plication to object recognition has not been yet explored in-depth.…”

Section: Introductionmentioning

confidence: 99%

“…By plugging our contributions in the BoW paradigm, we investigate how visual attention modeling can be applied to various steps in the processing pipeline. To the best of our knowledge, this is the first in-depth study about the application of visual saliency to object recognition with BoW approach at several stages, as: i) we extend the state-of-the-art on Saliencysensitive non-uniform feature sampling in a new Saliency-sensitive variableresolution feature space, ii) we introduce a completely new Saliency-Sensitive Coding of features and use the iii) Saliency-based feature pooling which has been shown to be efficient in referenced research [20,13].…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Perceptual modeling in the problem of active object recognition in visual scenes

González-Díaz

Buso

Benois-Pineau

2016

Pattern Recognition

View full text Add to dashboard Cite

Incorporating models of human perception into the process of scene interpretation and object recognition in visual content is a strong trend in computer vision. In this paper we tackle the modeling of visual perception via automatic visual saliency maps for object recognition. Visual saliency represents an efficient way to drive the scene analysis towards particular areas considered 'of interest' for a viewer and an efficient alternative to computationally intensive sliding window methods for object recognition. Using saliency maps, we consider biologically-inspired independent paths of central and peripheral vision and apply them to fundamental steps of the so-called Bag-of-Words (BoW) paradigm, such as features sampling, pooling and encoding. Our proposal has been evaluated addressing the challenging task of active object recognition, and the results show that our method not only improves the baselines, but also achieves stateof-the-art performance in various datasets at very competitive computational times.

show abstract

“…In our previous work [21] we have proposed an object recognition approach in the family of methods which use psycho-visual weighting [22] of the conventional Bag-of-Visual-Words (BoVW) [23], [24] paradigm. Here we compute SURF descriptors [25] on a dense grid, then a visual dictionary is built from all descriptors extracted in the training video database.…”

Section: Object Recognition Approachmentioning

confidence: 99%

Object recognition with top-down visual attention modeling for behavioral studies

Buso

González-Díaz

Benois-Pineau

2015

2015 IEEE International Conference on Image Processing (ICIP)

View full text Add to dashboard Cite

Behavioural analysis in instrumental activities of daily living has become a powerful tool in clinical studies and rises the question of what objects are manipulated by patients. In this paper we present a top-down probabilistic visual attention model for manipulated object recognition in egocentric video content. Although arms often occlude objects and are usually seen as a burden for many vision systems, they become an asset in our approach, as we extract both global and local features describing their geometric layout and pose, as well as the objects being manipulated. We integrate this information in a probabilistic generative model, provide update equations that automatically compute the model parameters optimizing the likelihood of the data, and design a method to generate maps of visual attention that are later used in an object-recognition framework. This taskdriven assessment reveals that the proposed method outperforms the state of the art in object recognition for egocentric video content.

show abstract

Spatial Locality Weighting of Features Using Saliency Map with a Bag-of-Visual-Words Approach

Cited by 13 publications

References 17 publications

Saliency Weighted Convolutional Features for Instance Search

Saliency Weighted Convolutional Features for Instance Search

Perceptual modeling in the problem of active object recognition in visual scenes

Object recognition with top-down visual attention modeling for behavioral studies

Contact Info

Product

Resources

About