Addressing delayed feedback for continuous training with neural networks in CTR prediction

Abstract

One of the challenges in display advertising is that the distribution of features and click through rate (CTR) can exhibit large shifts over time due to seasonality, changes to ad campaigns and other factors. The predominant strategy to keep up with these shifts is to train predictive models continuously, on fresh data, in order to prevent them from becoming stale. However, in many ad systems positive labels are only observed after a possibly long and random delay. These delayed labels pose a challenge to data freshness in continuous training: fresh data may not have complete label information at the time they are ingested by the training algorithm. Naive strategies which consider any data point a negative example until a positive label becomes available tend to underestimate CTR, resulting in inferior user experience and suboptimal performance for advertisers. The focus of this paper is to identify the best combination of loss functions and models that enable large-scale learning from a continuous stream of data in the presence of delayed labels. In this work, we compare 5 different loss functions, 3 of them applied to this problem for the first time. We benchmark their performance in offline settings on both public and proprietary datasets in conjunction with shallow and deep model architectures. We also discuss the engineering cost associated with implementing each loss function in a production environment. Finally, we carried out online experiments with the top performing methods, in order to validate their performance in a continuous training scheme. While training on 668 million in-house data points offline, our proposed methods outperform previous state-of-the-art by 3% relative cross entropy (RCE). During online experiments, we observed 55% gain in revenue per thousand requests (RPMq) against naive log loss.

Capturing Delayed Feedback in Conversion Rate Prediction via Elapsed-Time Sampling

“…Second, in most cases, FNC and FNW perform better than the vanilla baseline. Specially, FNW outperforms the baseline in both PR-AUC and NLL, which is consistent with the results reported in Ktena et al (2019). Third, existing methods show little superior performance in terms of AUC, while our method outperform the best baseline of 0.26% and 0.44% AUC scores on the public and anonymous dataset, respectively.…”

“…In order to capture the dynamic change of user needs, commercial systems often update learned models with update-to-date data within a short time, i.e., in an online training manner (Jugovac, Jannach, and Karimi 2018;Guo et al 2019;Ktena et al 2019). This further complicates the CVR prediction since conversions usually do not happen immediately after a user click.…”

User Response Prediction in Online Advertising

“…Recently, authors in [63] have centered their work on delayed positive feedback at stream media to study the effect of two factors, such as the trend and seasonality, in online advertising. In live streams, the predicting models are dealt with the cold start issue.…”

Deep Exploration for Recommendation Systems

“…Supervised learning procedures, such as collaborative filtering (Schafer et al 2007) and content-based filtering (Blanda 2016) that form the cornerstone of RSs, are typically used to model the probability that each piece of content will immediately engage a given user. However, those methods are inadequate when optimizing for delayed user feedback (Joulani, Gyorgy, and Szepesvári 2013;Ktena et al 2019). In particular, the value of a recommendation can become evident in later interactions with a user, rather than through immediate engagement.…”