The Visual Word Booster: A Spatial Layout of Words Descriptor Exploiting Contour Cues

Abstract

Although researchers have made efforts to use the spatial information of visual words to obtain better image representations, none of the studies take contour cues into account. Meanwhile, it has been shown that contour cues are important to the perception of imagery in the literature. Inspired by these studies, we propose to use the Spatial Layout of Words (SLoW) to boost visual word based image descriptors by exploiting contour cues. Essentially, the SLoW descriptor utilises contours and incorporates different types of commonly used visual words, including hand-crafted basic contour elements (referred to as "contons"), textons and Scale-Invariant Feature Transform (SIFT) words, deep convolutional words and a special type of words: LBP (Local Binary Pattern) codes. Moreover, SLoW features are combined with Spatial Pyramid Matching (SPM) or Vector of Locally Aggregated Descriptors (VLAD) features. The SLoW descriptor and its combined versions are tested in different tasks. Our results show that they are superior to, or at least comparable to, their counterparts examined in this study. In particular, the joint use of the SLoW descriptor boosts the performance of the SPM and VLAD descriptors. We attribute these results to the fact that contour cues are important to human visual perception and, the SLoW descriptor captures not only local image characteristics but also the global spatial layout of these characteristics in a more perceptually consistent way than its counterparts.

A Perception-Inspired Deep Learning Framework for Predicting Perceptual Texture Similarity

“…Extracting contour maps Previous study on visual perception [23], [24], [30] has shown that edge information plays an important role in the estimation of texture similarity. In order to learn the perceptual similarity between the two texture images, we therefore propose to use the contour maps of the texture images as auxiliary inputs for prediction.…”

“…In [24], Dong and Chantler observed that contour maps provide better texture representation than other types of local texture characteristics. They attributed this success to the long-range interactions between local image characteristics encoded by contours [23], [30].…”

“…This approach not only explores the advantages of CNNs but also benefits from contour cues which are able to encode long-range interaction. First, contour maps of the textures are extracted because of its importance to texture representation [30] [24]. Second, the paired texture images and the corresponding contour maps are used to train a similarity learning network.…”

Defect Detection and Classification by Training a Generic Convolutional Neural Network Encoder

“…Pre-trained CNNs can also be fine-tuned using a specific small image dataset [15]. On the other hand, Cimpoi et al [15] and Dong and Dong [34] used the features extracted at the convolutional layer of a pretrained classification CNN to learn visual words for image representation. In particular, it has been shown that the features extracted at the convolutional layer are less dependent of the dataset than those extracted at the fully-connected layer.…”

“…We did not tune the parameters of the linear SVM. The C value was set to 10 as performed in earlier studies [15], [34].…”