Fashion Meets Computer Vision and NLP at e-Commerce Search

Zoghbi, Susana; Heyman, Geert; Gómez, Juan Carlos; Moens, Marie‐Francine

doi:10.17706/ijcee.2016.8.1.31-43

Cited by 12 publications

(14 citation statements)

References 24 publications

(25 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Image search aims to retrieve relevant images to a textual or visual query. In [18], Zoghbi et al experiment with canonical correlation analysis (CCA) and bilingual latent Dirichlet allocation (BiLDA) to realize cross-modal fashion search from a textual query to images and from an image query to text. In [12], cross-modal search of fashion items is achieved with intermodal representations for visual and textual fashion attributes inferred with a neural network alignment model.…”

Section: Related Workmentioning

confidence: 99%

“…E-commerce product descriptions tend to be noisy and incomplete. To acquire the textual fashion attributes, we follow the approach of Zoghbi et al [18] and filter the product descriptions with the glossary of the online clothing shop Zappos 2 . This process only retains fashion-related phrases and removes most noise.…”

Section: Visual and Textual Attribute Detectionmentioning

confidence: 99%

“…We train our model on the Amazon Dresses dataset, which we obtained from Zoghbi et al [18]. This dataset consists of 53 689 image-text pairs, where each pair consists of an image of a dress and the corresponding product description.…”

Section: Experimental Setup 41 Datasetmentioning

confidence: 99%

“…Their corresponding texts roughly describe the image content and can be noisy. We use 48 689 image-text pairs for training, 4000 for validation and 1000 for testing in the same setup as in [18]. During testing, we evaluate the quality of the inferred intermodal representations in a multimodal retrieval setting.…”

Section: Experimental Setup 41 Datasetmentioning

confidence: 99%

See 3 more Smart Citations

Web Search of Fashion Items with Multimodal Querying

Laenen

Zoghbi

Moens

2018

Proceedings of the Eleventh ACM International Conference on Web Search and Data Mining

Self Cite

View full text Add to dashboard Cite

In this paper, we introduce a novel multimodal fashion search paradigm where e-commerce data is searched with a multimodal query composed of both an image and text. In this setting, the query image shows a fashion product that the user likes and the query text allows to change certain product attributes to fit the product to the user's desire. Multimodal search gives users the means to clearly express what they are looking for. This is in contrast to current e-commerce search mechanisms, which are cumbersome and often fail to grasp the customer's needs. Multimodal search requires intermodal representations of visual and textual fashion attributes which can be mixed and matched to form the user's desired product, and which have a mechanism to indicate when a visual and textual fashion attribute represent the same concept. With a neural network, we induce a common, multimodal space for visual and textual fashion attributes where their inner product measures their semantic similarity. We build a multimodal retrieval model which operates on the obtained intermodal representations and which ranks images based on their relevance to a multimodal query. We demonstrate that our model is able to retrieve images that both exhibit the necessary query image attributes and satisfy the query texts. Moreover, we show that our model substantially outperforms two state-of-the-art retrieval models adapted to multimodal fashion search.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Visual and Textual Attribute Detectionmentioning

confidence: 99%

Section: Experimental Setup 41 Datasetmentioning

confidence: 99%

Section: Experimental Setup 41 Datasetmentioning

confidence: 99%

See 2 more Smart Citations

Web Search of Fashion Items with Multimodal Querying

Laenen

Zoghbi

Moens

2018

Proceedings of the Eleventh ACM International Conference on Web Search and Data Mining

Self Cite

View full text Add to dashboard Cite

show abstract

“…In both cases the final aim is to recognize the image content in terms of depicted objects [7] or scene [5]. Previous work relies mostly on content-based strategies that either predict the text from the images by training a model of their relationship [16,17,18,19], or propagates tags through a k-nearest neighbor method [4,15]. In these studies, the analysis is generally performed on data collections where the ground truth image annotations are well defined with a large consensus among of users or experts.…”

Section: Related Workmentioning

confidence: 99%

What Would They Say? Predicting User's Comments in Pinterest

Gómez

Tommasi

Zoghbi

et al. 2016

IEEE Latin Am. Trans.

Self Cite

View full text Add to dashboard Cite

When we refer to an image that attracts our attention, it is natural to mention not only what is literally depicted in the image, but also the sentiments, thoughts and opinions that it invokes in ourselves. In this work we deviate from the standard mainstream tasks of associating tags or keywords to an image, or generating content image descriptions, and we introduce the novel task of automatically generate user comments for an image. We present a new dataset collected from the social media Pinterest and we propose a strategy based on building joint textual and visual user models, tailored to the specificity of the mentioned task. We conduct an extensive experimental analysis of our approach on both qualitative and quantitative terms, which allows assessing the value of the proposed approach and shows its encouraging results against several existing image-to-text methods.

show abstract

Predictive modeling of consumer color preference: Using retail data and merchandise images

Chen

Yang

et al. 2020

Journal of Forecasting

View full text Add to dashboard Cite

The popularity of a fashion item depends on its color, shape, texture, and price. For different items (with all attributes identical except color) of a specific product, fashion retailers need to learn consumer color preference and decide their order quantities accordingly to match their products to consumer demand. This study aims to predict consumer color preference using the knowledge learned from merchandise images, historical retail data, and fashion trends. In our work, merchandise images are analyzed to extract color features, and the retail data of a sportswear retailer are used to reveal consumer choices among items with various colors. Choice behavior is described by a multinomial logit model, whose utility function captures the relationship between color features and popularity. Both linear functions and neural networks are applied to represent the utility function, and their out-of-sample prediction performances are compared. According to the out-of-sample performance test, our model shows reasonable predictive power and can outperform order decisions made by fashion buyers.

show abstract

Fashion Meets Computer Vision and NLP at e-Commerce Search

Cited by 12 publications

References 24 publications

Web Search of Fashion Items with Multimodal Querying

Web Search of Fashion Items with Multimodal Querying

What Would They Say? Predicting User's Comments in Pinterest

Predictive modeling of consumer color preference: Using retail data and merchandise images

Contact Info

Product

Resources

About