An Ensemble-based Approach to Click-Through Rate Prediction for Promoted Listings at Etsy

Abstract

Etsy 1 is a global marketplace where people across the world connect to make, buy, and sell unique goods. Sellers at Etsy can promote their product listings via advertising campaigns similar to traditional sponsored search ads. Click-Through Rate (CTR) prediction is an integral part of online search advertising systems where it is utilized as an input to auctions which determine the final ranking of promoted listings to a particular user for each query. In this paper, we provide a holistic view of Etsy's promoted listings' CTR prediction system and propose an ensemble learning approach which is based on historical or behavioral signals for older listings, as well as content-based features for new listings. We obtain representations from texts and images by utilizing state-of-the-art deep learning techniques and employ multimodal learning to combine these different signals. We compare the system to non-trivial baselines on a large-scale, real world dataset from Etsy, demonstrating the effectiveness of the model and strong correlations between offline experiments and online performance. The paper is also the first technical overview to this kind of product in an e-commerce context. Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from permissions@acm.org. CCS CONCEPTS

Model-agnostic vs. Model-intrinsic Interpretability for Explainable Product Search

“…For example, Bi et al [7? ] extract fine-grained review information with embedding networks; Guo et al [24] model long/short term user preferences with attention networks over user query history. There are also considerable studies on extracting ranking features and applying learning-to-rank methods for product search [5,13,29,32,69]. In this paper, our main focus is not to build the state-of-the-art product search models but to explore how to build effective search explanations to better improve user experience.…”

Explainable Product Search with a Dynamic Relation Embedding Model

“…The Latent Semantic Entity model (LSE) is the first latent space model proposed for product search by Van Gysel et al [57]. It encodes queries and n-grams with a non-linear projection function similar to Equation (7). It also learns the embedding representations of items by maximizing the similarity between an item and the encoded n-grams extracted from the corresponding item reviews.…”

“…Zhang et al [73] and Bi et al [9] also extract product aspects from review data and build embedding networks that encodes both items and their extracted aspects for conversational product search and recommendation. Besides latent embedding techniques, there is a variety of studies [7,28,30] on extracting different text and product features and feeding them into learning-to-rank models for the optimization of different product retrieval objectives. Wu et al [61] manually extract multiple statistic features from product search logs and construct an ensemble tree model to predict user clicks.…”

“…where Search&Purchase (u, q, i) is the only dynamic relation in D (u,v,i) . In DREM, u, i and embeddings for all other entities are jointly learned with the parameters W and b in Equation (7). To conduct product search for a specific user u with query q, we simply rank products i ∈ I with their estimated purchase probability P(i |u, v).…”

“…On Cell Phones & Accessories, DREM Bnd significantly outperformed DREM N oMet a with a 25% improvement on MAP. According to a recent report 7 , most people have high loyalty to the manufacturer of their phones and 56% of people who currently possess a smartphone used to own a phone from the same manufacturer. Thus, it is not surprising to see that Is_brand exhibits high correlations with user purchases in Cell Phones & Accessories.…”