A contextual-bandit approach to personalized news article recommendation

Abstract

Personalized web services strive to adapt their services (advertisements, news articles, etc.) to individual users by making use of both content and user information. Despite a few recent advances, this problem remains challenging for at least two reasons. First, web service is featured with dynamically changing pools of content, rendering traditional collaborative filtering methods inapplicable. Second, the scale of most web services of practical interest calls for solutions that are both fast in learning and computation.In this work, we model personalized recommendation of news articles as a contextual bandit problem, a principled approach in which a learning algorithm sequentially selects articles to serve users based on contextual information about the users and articles, while simultaneously adapting its article-selection strategy based on user-click feedback to maximize total user clicks.The contributions of this work are three-fold. First, we propose a new, general contextual bandit algorithm that is computationally efficient and well motivated from learning theory. Second, we argue that any bandit algorithm can be reliably evaluated offline using previously recorded random traffic. Finally, using this offline evaluation method, we successfully applied our new algorithm to a Yahoo! Front Page Today Module dataset containing over 33 million events. Results showed a 12.5% click lift compared to a standard context-free bandit algorithm, and the advantage becomes even greater when data gets more scarce.

Contextual Search: A Computational Framework

“…A significant drop in the average expected click-through rate at the early stage has also been observed by Li et al [114], whose authors point out that like most feedback-based approaches, click-through data are usually more reliable with large amounts of historical data and less so for the ones with little or no history. Similarly, click-through data effectiveness can be improved if these data are combined with high click-through rate queries given that historic and low click-through rate queries are detrimental to the accuracy.…”

“…The problem of identifying breaking news is addressed by Li et al [114] where the authors address the problem using a multi-armed bandit model, that is, a statistical model sequentially selecting news based on the information of the user and news, and adapting itself on the basis of the user's click-through data in the same way a player who selects one arm of a bandit out of the possible arms receives a payoff and sequentially learns how to select the next arm for maximizing the total payoff.…”

CLEF 2017 NewsREEL Overview: A Stream-Based Recommender Task for Evaluation and Education

“…Subsequently, multiple updates of the data set have been released in scope of CLEF NewsREEL [13,18,15]. Li et al [19] model news recommendation as contextual bandit problem. They define an evaluation procedure yielding valid results in offline settings.…”

Gathering Additional Feedback on Search Results by Multi-Armed Bandits with Respect to Production Ranking

“…As long as arms here are vectors, these approaches [35,28] apply boosting rather than standard bandit algorithms to find an optimal value of ω. The second interpretation [22,25] considers a querydocument as an arm, the fact of an arm trial consists in two conditions: the corresponding document was presented on SERP and was examined by the user, and the reward is a click (reward=1) or a skip (reward=0). By assuming that the click probability linearly depends on a number of features of the query-document pair, one is able to estimate this probability and confidence in this estimate for each pair.…”