Contextual Combinatorial Bandit and its Application on Diversified Online Recommendation

Abstract

Recommender systems are faced with new challenges that are beyond traditional techniques. For example, most traditional techniques are based on similarity or overlap among existing data, however, there may not exist sufficient historical records for some new users to predict their preference, or users can hold diverse interest, but the similarity based methods may probably over-narrow it.To address the above challenges, we develop a principled approach called contextual combinatorial bandit in which a learning algorithm can dynamically identify diverse items that interest a new user. Specifically, each item is represented as a feature vector, and each user is represented as an unknown preference vector. On each of n rounds, the bandit algorithm sequentially selects a set of items according to the item-selection strategy that balances exploration and exploitation, and collects the user feedback on these selected items. A reward function is further designed to measure the quality (e.g. relevance or diversity) of the selected set based on observed feedback, and the goal of the algorithm is to maximize the total rewards of n rounds. The reward function only needs to satisfy two mild assumptions that is general enough to accommodate a large class of nonlinear functions. To solve this bandit problem, we provide algorithm that achievesÕ( √ n) regret after playing n rounds. Experiments conducted on real-wold movie recommendation dataset demonstrate that our approach can effectively address the above challenges and hence improve the performance of recommendation task.

An Arm-Wise Randomization Approach to Combinatorial Linear Semi-Bandits

“…The state-of-the-art C 2 UCB algorithm [9] solves the CLS problem while theoretically guaranteeing a sublinear regret bound. Its procedure is described in Algorithm 1.…”

“…Moreover, we suppose that the numbers of clusters and feature vectors belonging to a cluster are sufficiently larger than T and k, respectively, in clustered cases. For simplicity, we assume S t = {I ⊆ [N ] | |I| = k} Algorithm 1 C 2 UCB [9] and Perturbed C 2 UCB Input: λ > 0 and α t > 0 λ > 0, α t > 0 and c > 0 .…”

“…More recently, the combinatorial linear semi-bandit (CLS) problem has been studied as a generalization of both LB and CS for more complex and realistic applications [9], [10]. For example, the semi-bandit setting allows CLS to optimize recommender systems that display a set of items in each time window.…”

A framework for diversifying recommendation lists by user interest expansion

“…Noia et al [28] modeled users’ propensity toward selecting diverse items, and then presented a diversification method to re-rank the list of top-N items predicted by a recommendation algorithm. Qin et al [29] proposed a novel approach called contextual combinatorial bandit, where diverse items that may interest a new user can be dynamically identified by a learning algorithm. However, no specific diversity measure metric was used in the evaluation.…”

“…In our strategy, we also take into account the balance between diversity and accuracy. Additionally, some studies deal with the relation between diversity and accuracy in a unified model [20,29–31]. Compared with the unified model, one advantage of our strategy is that it can combine with different traditional RS without modifying them and leverage their advantages.…”