Online Interactive Collaborative Filtering Using Multi-Armed Bandit with Dependent Arms

Abstract

Online interactive recommender systems strive to promptly suggest to consumers appropriate items (e.g., movies, news articles) according to the current context including both the consumer and item content information. However, such context information is o en unavailable in practice for the recommendation, where only the users' interaction data on items can be utilized. Moreover, the lack of interaction records, especially for new users and items, worsens the performance of recommendation further. To address these issues, collaborative ltering (CF), one of the recommendation techniques relying on the interaction data only, as well as the online multi-armed bandit mechanisms, capable of achieving the balance between exploitation and exploration, are adopted in the online interactive recommendation se ings, by assuming independent items (i.e., arms). Nonetheless, the assumption rarely holds in reality, since the real-world items tend to be correlated with each other (e.g., two articles with similar topics).In this paper, we study online interactive collaborative ltering problems by considering the dependencies among items. We explicitly formulate the item dependencies as the clusters on arms, where the arms within a single cluster share the similar latent topics. In light of the topic modeling techniques, we come up with a generative model to generate the items from their underlying topics. Furthermore, an e cient online algorithm based on particle learning is developed for inferring both latent parameters and states of our model. Additionally, our inferred model can be naturally integrated with existing multi-armed selection strategies in the online interactive collaborating se ing. Empirical studies on two real-world applications, online recommendations of movies and news, demonstrate both the e ectiveness and e ciency of the proposed approach.

Interactive Anomaly Detection on Attributed Networks

“…On the other hand, CoLin [50] assumes that the reward in bandit is generated through an additive model, indicating that friends' feedback on their recommendations can be passed via the network to explain the target user's feedback. Recently, researchers also tried to capture the arm dependency by organizing different arms into different clusters [36,48] and it is very similar to our studied problem. However, simply performing clustering on attributed networks may ignore the inherent node dependencies and may lead to suboptimal results in interactive anomaly discovery.…”

Soliciting User Preferences in Conversational Recommender Systems via Usage-related Questions

“…In the early studies of CRSs it was common to ask for users opinions on an item itself (Zhao et al, 2013;Wang et al, 2017). These approaches usually combine the features of static recommender systems such as CF with user interaction in real-time.…”

Learning the Truth in Social Networks Using Multi-Armed Bandit

“…In [28], the authors studied online collaborative filtering with dependent arms, where they combined particle filtering with the upper confidence bound (UCB) algorithm and Thompson sampling. The limitation of the work is that it does not consider adversarial effects.…”