Deep Relative Attributes

Yang, Xiaoshan; Zhang, Tianzhu; Xu, Changsheng; Yan, Shuicheng; Hossain, M. Shamim; Ghoneim, Ahmed

doi:10.1109/tmm.2016.2582379

Cited by 88 publications

(53 citation statements)

References 49 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…for clothing; and beakshape, wings-pattern, tail-length etc. for birds, which have ... been found instrumental in describing images and objects in a semantic context [25], [27], [55], [56]. Formally, let a = [a 1 , a 2 , .…”

Section: B Attribute Prediction Taskmentioning

confidence: 99%

Dynamically Modulated Deep Metric Learning for Visual Search

Manandhar

Baştan

Yap

2020

ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

View full text Add to dashboard Cite

Deep metric learning applied to various multimedia applications has shown promising results in tasks such identification, retrieval and recognition. Existing metric learning methods often do not consider different granularity in visual similarity. However, in many domain applications, images exhibit similarity at multiple granularities with visual semantic concepts, e.g. fashion demonstrates similarity ranging from clothing of the exact same instance to similar looks/design or a common category. Therefore, training image triplets/pairs used for metric learning inherently possess different degree of information. However, the existing methods often treats them with equal importance during training. This hinders capturing the underlying granularities in feature similarity which is required for effective visual search.In view of this, we propose a new deep semantic granularity metric learning (SGML) that develops a novel idea of detecting and leveraging attribute semantic space to capture different granularity of similarity, and then integrate this information into deep metric learning. The proposed framework simultaneously learns image attributes and embeddings using multitask CNNs with shared parameters. The two tasks are not only jointly optimized but are further linked by the semantic granularity similarity mappings to leverage the correlations between the tasks. To this end, we propose a new soft-binomial deviance loss that effectively integrates the degree of information in training samples, which helps to capture visual similarity at multiple granularities. Compared to recent ensemble-based methods, our framework is conceptually elegant, computationally simple and provides better performance. We perform extensive experiments on benchmark metric learning datasets and demonstrate that our method outperforms recent state-of-the-art methods, e.g. , 1-4.5% improvement in Recall@1 over the previous state-of-the-arts [1], [2] on DeepFashion In-Shop dataset.

show abstract

Section: B Attribute Prediction Taskmentioning

confidence: 99%

Dynamically Modulated Deep Metric Learning for Visual Search

Manandhar

Baştan

Yap

2020

ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

View full text Add to dashboard Cite

show abstract

“…Attribute Comparisons: Relative attributes [9] model visual properties in a comparative manner, and a variety of applications have been explored in online shopping [16], fashion [17], biometrics [18], and graphical design [19]. Recent work explores novel learning schemes to train relative attributes accurately, including new deep network architectures [11], [12], [13], part discovery [10], local learning [4], [20], and multi-task approaches [21]. Our contribution is an approach to actively elicit training examples for ranking, which could facilitate training for many of the above formulations.…”

Section: Related Workmentioning

confidence: 99%

“…task [9], [10], [11], [12], [13]: given two images, the system must infer which one more evidently exhibits an attribute (e.g., which is more furry, more red, more smiling). See Figure 1.…”

Section: Introductionmentioning

confidence: 99%

Thinking Outside the Pool: Active Training Image Creation for Relative Attributes

Grauman

2019

2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

View full text Add to dashboard Cite

Current wisdom suggests more labeled image data is always better, and obtaining labels is the bottleneck. Yet curating a pool of sufficiently diverse and informative images is itself a challenge. In particular, training image curation is problematic for fine-grained attributes, where the subtle visual differences of interest may be rare within traditional image sources. We propose an active image generation approach to address this issue. The main idea is to jointly learn the attribute ranking task while also learning to generate novel realistic image samples that will benefit that task. We introduce an end-to-end framework that dynamically "imagines" image pairs that would confuse the current model, presents them to human annotators for labeling, then improves the predictive model with the new examples. With results on two datasets, we show that by thinking outside the pool of real images, our approach gains generalization accuracy for challenging fine-grained attribute comparisons.

show abstract

“…Relative attributes, first introduced in [34], represent an image's attribute strength with respect to other images [7,10,20,29,37,40,41,[45][46][47][48], and are a richer representation than binary presence/absence. Relative attributes enable visual comparisons between images (e.g., the left shoe is more sporty than the right), and have been used to discern fine-grained differences [46,47] and predict image virality [7].…”

Section: Relative Attributesmentioning

confidence: 99%

“…Relative attributes enable visual comparisons between images (e.g., the left shoe is more sporty than the right), and have been used to discern fine-grained differences [46,47] and predict image virality [7]. Recently, deep CNNs have been used to both predict relative attributes [40,41,45] as well as generate synthetic images of varying attribute strengths [44,48]. However, no prior work considers which relative attributes stand out, or what relative attributes humans tend to use in speech.…”

Section: Relative Attributesmentioning

confidence: 99%

Compare and Contrast: Learning Prominent Visual Differences

Chen

Grauman

2018

2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition

View full text Add to dashboard Cite

Relative attribute models can compare images in terms of all detected properties or attributes, exhaustively predicting which image is fancier, more natural, and so on without any regard to ordering. However, when humans compare images, certain differences will naturally stick out and come to mind first. These most noticeable differences, or prominent differences, are likely to be described first. In addition, many differences, although present, may not be mentioned at all. In this work, we introduce and model prominent differences, a rich new functionality for comparing images. We collect instance-level annotations of most noticeable differences, and build a model trained on relative attribute features that predicts prominent differences for unseen pairs. We test our model on the challenging UT-Zap50K shoes and LFW10 faces datasets, and outperform an array of baseline methods. We then demonstrate how our prominence model improves two vision tasks, image search and description generation, enabling more natural communication between people and vision systems.

show abstract

Deep Relative Attributes

Cited by 88 publications

References 49 publications

Dynamically Modulated Deep Metric Learning for Visual Search

Dynamically Modulated Deep Metric Learning for Visual Search

Thinking Outside the Pool: Active Training Image Creation for Relative Attributes

Compare and Contrast: Learning Prominent Visual Differences

Contact Info

Product

Resources

About