A Hybrid Bandit Model with Visual Priors for Creative Ranking in Display Advertising

Abstract

Creative plays a great important role in e-commerce for exhibiting products. Sellers usually create multiple creatives for comprehensive demonstrations, thus it is crucial to display the most appealing design to maximize the Click-Through Rate (CTR). For this purpose, modern recommender systems dynamically rank creatives when a product is proposed for a user. However, this task suffers more cold-start problem than conventional products recommendation since the user-click data is more scarce and creatives potentially change more frequently. In this paper, we propose a hybrid bandit model with visual priors which first makes predictions with a visual evaluation, and then naturally evolves to focus on the specialities through the hybrid bandit model. Our contributions are three-fold: 1) We present a visual-aware ranking model (called VAM) that incorporates a list-wise ranking loss for ordering the creatives according to the visual appearance. 2) Regarding visual evaluation as a prior, the hybrid bandit model (called HBM) is proposed to evolve consistently to make better posteriori estimations by taking more observations into consideration for online scenarios. 3) A first large-scale creative dataset, CreativeRanking 1 , is constructed, which contains over 1.7M creatives of 500k products as well as their real impression and click data. Extensive experiments have also been conducted on both our dataset and public Mushroom dataset, demonstrating the effectiveness of the proposed method.

Convolutional Neural Bandit for Visual-aware Recommendation

“…Riquelme et al [2018] used L-layer DNN to learn a representation for each arm and applies Thompson sampling on the low-dimension embeddings for exploration. Wang et al [2021] extended the above framework to visual advertising by using CNN to train arm embeddings. However, they did not provide the regret analysis.…”

“…The contextual Multi-Armed Bandit (MAB) can naturally formulate the procedure and has shown success in online advertising [Li et al, 2010;Wang et al, 2021;Li et al, 2016a;. In the standard bandit setting, suppose there are n arms (images) in a round, each of which is represented by a feature vector or matrix, the learner needs to pull an arm and then observe the reward (CTR).…”

“…Zhang et al [2020] and both used a fully-connected neural network to learn the reward function; the former adopted the Thompson sampling and the latter adopted the Upper Confidence Bound (UCB) strategies for exploration. To further improve the performance specifically for visual recommendation, Wang et al [2021] used Convolutional Neural Networks (CNN) to train embeddings of images and then fed it into the Thompson sampling bandit based on the framework of Riquelme et al [2018]. However, these two approaches did not provide analysis regarding the regret bound.…”

“…In this paper, we focus on visual advertising and propose to use convolutional neural networks to learn the reward function with UCB-based exploration in contextual bandits. Note that Wang et al [2021] used CNN to train the embeddings for arms instead of learning the reward function. Inspired by recent advances on the convergence theory in over-parameterized neural networks Allen-Zhu et al, 2019], we build strong connections among the network function, Convolutional Neural Tangent Kernel (CNTK) [Jacot et al, 2018;Arora et al, 2019], and ridge regression to prove an upper confidence bound and the regret bound.…”