From Street Photos to Fashion Trends: Leveraging User-Provided Noisy Labels for Fashion Understanding

Abstract

There is increased interest in using street photos to understand fashion trends. Though street photos usually contain rich clothing information, there are several technical challenges to their analysis. First, street photos collected from social media sites often contain user-provided noisy labels, and training models using these labels may deteriorate prediction performance. Second, most existing methods predict multiple clothing attributes individually and do not consider the potential to share knowledge between related tasks. In addition to these technical challenges, most fashion image datasets created by previous studies focus on American and European fashion styles. To address these technical challenges and understand fashion trends in Asia, we created RichWear, a new street fashion dataset containing 322,198 images with various text labels for fashion analysis. This dataset, collected from an Asian social network site, focuses on street styles in Japan and other Asian areas. RichWear provides a subset of expert-verified labels in addition to user-provided noisy labels for model training and evaluation. We propose the Fashion Attributes Recognition Network (FARNet) based on the multi-task learning framework to improve fashion recognition. Instead of predicting each clothing attribute individually, FARNet predicts three types of attributes simultaneously, and, once trained, this network leverages the noisy labels and generates corrected labels based on the input images. Experimental results show that this approach significantly outperforms existing methods. Applying the trained model to the RichWear dataset, we report Asian fashion trends and street styles based on predicted labels and image clusters from latent feature vectors.

A Fine-grained Attributes Recognition Model for Clothing Based on Improved the CSPDarknet and PAFPN Network

“…In recent years, due to the rapid development of apparel e-commerce, there is a wide market demand for ne-grained attributes recognition of apparel images. At the same time, the recognition of clothing attributes is one of the fundamental issues in subsequent intelligent analysis tasks, which has important research signi cance as it can promote the completion of important tasks such as trend prediction [4] and retrieval [5]. Dresses are a very fashionable category in consumer daily clothing purchases.…”

Deep Learning for Clothing Style Recognition Using YOLOv5

“…In [ 42 ], data mining and symmetry-based learning techniques were addressed to create a classification model for predicting the garment category. Based on a fashion attributes recognition network, the multi-task learning framework to improve fashion recognition was proposed to leverage the noisy labels and generate corrected labels [ 43 ]. In [ 44 ], both deep learning and image processing techniques were applied to automatically recognize and classify logos, stripes, colors, and other features of clothing.…”