Generalising Fine-Grained Sketch-Based Image Retrieval

Pang, Kaiyue; Li, Ké; Yang, Yongxin; Zhang, Honggang; Hospedales, Timothy M.; Xiang, Tao; Song, Yi-Zhe

doi:10.1109/cvpr.2019.00077

Cited by 97 publications

(66 citation statements)

References 27 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Most VPR problems are considered retrieval problems as well. In the literature, various methods have been developed for image retrieval, including text-based methods, content-based methods [33][34][35][36][37][38], sketch-based methods [39][40][41], and semantic-based methods [36,41,42]. Overall, most of the studies focus on feature extraction and representation.…”

Section: Image Retrievalmentioning

confidence: 99%

GSAP: A Global Structure Attention Pooling Method for Graph-Based Visual Place Recognition

Yang

Liu

et al. 2021

Remote Sensing

View full text Add to dashboard Cite

The Visual Place Recognition problem aims to use an image to recognize the location that has been visited before. In most of the scenes revisited, the appearance and view are drastically different. Most previous works focus on the 2-D image-based deep learning method. However, the convolutional features are not robust enough to the challenging scenes mentioned above. In this paper, in order to take advantage of the information that helps the Visual Place Recognition task in these challenging scenes, we propose a new graph construction approach to extract the useful information from an RGB image and a depth image and fuse them in graph data. Then, we deal with the Visual Place Recognition problem as a graph classification problem. We propose a new Global Pooling method—Global Structure Attention Pooling (GSAP), which improves the classification accuracy by improving the expression ability of the Global Pooling component. The experiments show that our GSAP method improves the accuracy of graph classification by approximately 2–5%, the graph construction method improves the accuracy of graph classification by approximately 4–6%, and that the whole Visual Place Recognition model is robust to appearance change and view change.

show abstract

Section: Image Retrievalmentioning

confidence: 99%

GSAP: A Global Structure Attention Pooling Method for Graph-Based Visual Place Recognition

Yang

Liu

et al. 2021

Remote Sensing

View full text Add to dashboard Cite

show abstract

“…Furthermore, a spatially aware model which combines coarse and fine semantic information is proposed by [27]. Pang et al [24] identify cross-category generalization for FG-SBIR as a domain generalization problem and propose an unsupervised learning approach to modeling a universal manifold of prototypical visual sketch traits. Though FG-SBIR research works achieve inspiring progress, they focus on retrieving a single object, which may not fit well to SBIR applications in real scenarios.…”

Section: Related Workmentioning

confidence: 99%

SceneSketcher: Fine-Grained Image Retrieval with Scene Sketches

Liu

Zhang

Deng

et al. 2020

Lecture Notes in Computer Science

View full text Add to dashboard Cite

“…Recently, the research related to sketch has flourished. Up to now, many problems have been studied, including sketch recognition [6,7], sketchbased image retrieval (SBIR) [8,9], and sketch-based 3D model retrieval [10], just to name a few. What is more, sketch-based fashion image retrieval is still relatively new.…”

Section: Introductionmentioning

confidence: 99%

A New Algorithm for Sketch‐Based Fashion Image Retrieval Based on Cross‐Domain Transformation

Lei

Chen

Wang

et al. 2021

Wireless Communications and Mobile Computing

View full text Add to dashboard Cite

Due to the rise of e-commerce platforms, online shopping has become a trend. However, the current mainstream retrieval methods are still limited to using text or exemplar images as input. For huge commodity databases, it remains a long-standing unsolved problem for users to find the interested products quickly. Different from the traditional text-based and exemplar-based image retrieval techniques, sketch-based image retrieval (SBIR) provides a more intuitive and natural way for users to specify their search need. Due to the large cross-domain discrepancy between the free-hand sketch and fashion images, retrieving fashion images by sketches is a significantly challenging task. In this work, we propose a new algorithm for sketch-based fashion image retrieval based on cross-domain transformation. In our approach, the sketch and photo are first transformed into the same domain. Then, the sketch domain similarity and the photo domain similarity are calculated, respectively, and fused to improve the retrieval accuracy of fashion images. Moreover, the existing fashion image datasets mostly contain photos only and rarely contain the sketch-photo pairs. Thus, we contribute a fine-grained sketch-based fashion image retrieval dataset, which includes 36,074 sketch-photo pairs. Specifically, when retrieving on our Fashion Image dataset, the accuracy of our model ranks the correct match at the top-1 which is 96.6%, 92.1%, 91.0%, and 90.5% for clothes, pants, skirts, and shoes, respectively. Extensive experiments conducted on our dataset and two fine-grained instance-level datasets, i.e., QMUL-shoes and QMUL-chairs, show that our model has achieved a better performance than other existing methods.

show abstract

Generalising Fine-Grained Sketch-Based Image Retrieval

Cited by 97 publications

References 27 publications

GSAP: A Global Structure Attention Pooling Method for Graph-Based Visual Place Recognition

GSAP: A Global Structure Attention Pooling Method for Graph-Based Visual Place Recognition

SceneSketcher: Fine-Grained Image Retrieval with Scene Sketches

A New Algorithm for Sketch‐Based Fashion Image Retrieval Based on Cross‐Domain Transformation

Contact Info

Product

Resources

About