Multi-Modal and Multi-Domain Embedding Learning for Fashion Retrieval and Analysis

Gu, Xiaoling; Wong, Yongkang; Shou, Lidan; Peng, Pai; Chen, Gang; Kankanhalli, Mohan

doi:10.1109/tmm.2018.2876822

Cited by 39 publications

(15 citation statements)

References 37 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Reasoning about the similarity between images or data of different modalities is an inherent challenge in computer vision. Beyond its prevalence in fundamental problems such as image-sentence retrieval [41,38], cross-domain image-matching [32,16], attribution learning [4,33] and visual categorization [29], it also has an increasingly prominent role in computer vision problems in the fashion and retail domains like outfit style modeling [14], fashion item retrieval and recommendation [10,22] and automatic capsule wardrobe generation [15]. Metric learning (the task of learning a distance function between features based on supervised similar/dissimilar pairs) is a common approach 1 https://github.com/rxtan2/ Learning-Similarity-Conditions Figure 1: We propose the SCE-Net model for learning multi-faceted similarity between images, such as compatibility of two fashion items.…”

Section: Introductionmentioning

confidence: 99%

Learning Similarity Conditions Without Explicit Supervision

Tan

Vasileva

Saenko

et al. 2019

2019 IEEE/CVF International Conference on Computer Vision (ICCV)

117

View full text Add to dashboard Cite

Many real-world tasks require models to compare images along multiple similarity conditions (e.g. similarity in color, category or shape). Existing methods often reason about these complex similarity relationships by learning condition-aware embeddings. While such embeddings aid models in learning different notions of similarity, they also limit their capability to generalize to unseen categories since they require explicit labels at test time. To address this deficiency, we propose an approach that jointly learns representations for the different similarity conditions and their contributions as a latent variable without explicit supervision. Comprehensive experiments 1 across three datasets, Polyvore-Outfits, Maryland-Polyvore and UT-Zappos50k, demonstrate the effectiveness of our approach: our model outperforms the state-of-the-art methods, even those that are strongly supervised with pre-defined similarity conditions, on fill-in-the-blank, outfit compatibility prediction and triplet prediction tasks. Finally, we show that our model learns different visually-relevant semantic sub-spaces that allow it to generalize well to unseen categories.

show abstract

Section: Introductionmentioning

confidence: 99%

Learning Similarity Conditions Without Explicit Supervision

Tan

Vasileva

Saenko

et al. 2019

2019 IEEE/CVF International Conference on Computer Vision (ICCV)

117

View full text Add to dashboard Cite

show abstract

“…In fashion recommendation, for example, Li et al [21] score and recommend fashion outfit candidates based on the appearances and metadata, Han et al [11] learn a visual-semantic space to not only perform the aforementioned recommendations but also predict the compatibility of a given outfit, Chen et al [5] propose a novel neural architecture for fashion recommendation based on both image region-level features and user review information, Wu et al [39] propose a visual and textual jointly enhanced interpretable model for fashion recommendations. In fashion retrieval, Liao et al [22] utilize an EI-tree which can cooperate with deep models for end-toend multimodal learning, Gu et al [8], Guo et al [9], Yuan and Lam [42] propose a multimodal framework for fashion analysis and data retrieval, Ma et al [25] forecast the fashion trend by a knowledge enhanced recurrent network model.…”

Section: Multimodality In Fashion Related Tasksmentioning

confidence: 99%

Sentiment Analysis of Fashion Related Posts in Social Media

Yuan

Lam

2022

Proceedings of the Fifteenth ACM International Conference on Web Search and Data Mining

View full text Add to dashboard Cite

The role of social media in fashion industry has been blooming as the years have continued on. In this work, we investigate sentiment analysis for fashion related posts in social media platforms. There are two main challenges of this task. On the first place, information of different modalities must be jointly considered to make the final predictions. On the second place, some unique fashion related attributes should be taken into account. While most existing works focus on traditional multimodal sentiment analysis, they always fail to exploit the fashion related attributes in this task. We propose a novel framework that jointly leverages the image vision, post text, as well as fashion attribute modality to determine the sentiment category. One characteristic of our model is that it extracts fashion attributes and integrates them with the image vision information for effective representation. Furthermore, it exploits the mutual relationship between the fashion attributes and the post texts via a mutual attention mechanism. Since there is no existing dataset suitable for this task, we prepare a large-scale sentiment analysis dataset of over 12k fashion related social media posts. Extensive experiments are conducted to demonstrate the effectiveness of our model. CCS CONCEPTS• Information systems → Sentiment analysis.

show abstract

“…Fashion design based on big data is now a widely used approach that greatly improves the success of design. 25…”

Section: Fashion Design Based On Big Datamentioning

confidence: 99%

Big data and digital design models for fashion design

Zhao

Liu

Zhao

2021

Journal of Engineered Fibers and Fabrics

View full text Add to dashboard Cite

The demand for fashion, and for virtual fitting and personalized fashion among customers, is changing the design and consumption of fashion. To meet such challenges, fashion design models are being developed based on big data and digitization, in which fashion is designed based on data, virtual fitting, design-support systems, and recommendation systems. This paper reviews the fashion design models proposed in recent years and considers future development directions for fashion design. Using big data and digital processing technologies, fashion designers identify the characteristics of popular fashions in the market, predict fashion trends, and create designs accordingly. The virtual fitting of scanatar, parametric mannequin, or even real human bodies, enables customers to quickly and easily find fashion that best meets their tastes and requirements. On consumer design-support platforms, consumers can freely select styles, colors, materials, and other fashion aspects and view the design output. Furthermore, fashion recommendation systems, guided by fashion design experts, have greatly improved consumer satisfaction with fashion design. Yet, current fashion design systems do not fully consider the performance of textile materials and do not involve functional fashion design, let alone comfort. Such limitations provide directions future research in fashion design.

show abstract

Multi-Modal and Multi-Domain Embedding Learning for Fashion Retrieval and Analysis

Cited by 39 publications

References 37 publications

Learning Similarity Conditions Without Explicit Supervision

Learning Similarity Conditions Without Explicit Supervision

Sentiment Analysis of Fashion Related Posts in Social Media

Big data and digital design models for fashion design

Contact Info

Product

Resources

About