Well Googled is Half Done: Multimodal Forecasting of New Fashion Product Sales with Image-based Google Trends

Abstract

This paper investigates the effectiveness of systematically probing Google Trends against textual translations of visual aspects as exogenous knowledge to predict the sales of brand-new fashion items, where past sales data is not available, but only an image and few metadata are available. In particular, we propose GTM-Transformer, standing for Google Trends Multimodal Transformer, whose encoder works on the representation of the exogenous time series, while the decoder forecasts the sales using the Google Trends encoding, and the available visual and metadata information. Our model works in a non-autoregressive manner, avoiding the compounding effect of the first-step errors. As a second contribution, we present the VISUELLE dataset, which is the first publicly available dataset for the task of new fashion product sales forecasting, containing the sales of 5577 new products sold between 2016-2019, derived from genuine historical data of Nunalie, an Italian fast-fashion company. Our dataset is equipped with images of products, metadata, related sales, and associated Google Trends. We use VISUELLE to compare our approach against state-of-the-art alternatives and numerous baselines, showing that GTM-Transformer is the most accurate in terms of both percentage and absolute error. It is worth noting that the addition of exogenous knowledge boosts the forecasting accuracy by 1.5% WAPE wise, showing the importance of exploiting Google Trends. The code and dataset are both available at https://github.com/HumaticsLAB/GTM-Transformer

Multimodal Quasi-AutoRegression: Forecasting the visual popularity of new fashion products

“…al. [8] criticised the reliance on purely AR networks for new product sale forecasting because of the compounding effect caused by first-step errors. Instead, they propose GTM-Transformer, a multi-modal, non-AR Transformer that utilizes images, text and time series of the garment's attributes collected from Google Trends.…”

“…Apart from the visual features of the garment, both [1] and [8] used fashion attributes as textual information. Fashion attributes can be extracted from images by specialised classification models without requiring manual annotation from the fashion designers and can offer valuable information to the overall neural network.…”

“…We therefore split the data into established products which are used as the training set and new products which are split in half for the validation and testing sets. We apply this protocol on Mallzee, SHIFT15m and Amazon Reviews datasets but for VISUELLE we follow the experimental protocol described in [8].…”

“…VISUELLE 3 is a public fashion image dataset that contains 12 week long sales sequences for 5577 garments spanning from October 2016 to December 2019 [8]. For each garment, it provides an image, textual information (categorical labels related to fashion categories, fabrics, colors) and time series related to the categorical labels collected from Google Trends.…”

“…However, new products by definition lack historical data which renders the use of conventional AR networks impracticable. Few recent research works have addressed sales forecasting of new garments, by utilizing KNN-based (nearest neighbors) [7], auto-regressive networks with auxiliary features (images, fashion attributes and events) [1] or a non-AR Transformer modelling images and fashion attributes along with the "target" time series of those attributes collected from Google Trends 1 [8]. However, fashion attributes are not always independent of each other.…”