Fast distributed bandits for online recommendation systems

Mahadik, Kanak; Wu, Qingyun; Li, Shuai; Sabne, Amit

doi:10.1145/3392717.3392748

“…Moreover, HMAB aims to learn latent parameters for the (1) , 𝑢 (2) , • • • , 𝑢 (𝑀 ) } A Arm set, A = {𝑎 (1) , 𝑎 (2) , • • • , 𝑎 (𝐾 ) } I Item set, I = {𝑖 (1) , 𝑖 (2) nodes in the hierarchy tree, which is totally different from our goal of exploring users' latent interests. Distributed bandit algorithms, such as DCCB [14] and DistCLUB [20], aim to speed up the computation by distributing the workloads in parallel. However, these methods do not address the issue of searching from tremendous items, the computational cost is still too expensive for responding users' requests in an online manner (for example, how to response 100 users' concurrent requests within 10 milliseconds in a scenario involving one million items).…”

Section: Cluster-of-bandit Algorithmsmentioning

confidence: 99%

Show Me the Whole World

Song

¹

,

Sun

²

,

Lian

³

et al. 2022

Proceedings of the Fifteenth ACM International Conference on Web Search and Data Mining

10

0

View full text Add to dashboard Cite

User interest exploration is an important and challenging topic in recommender systems, which alleviates the closed-loop effects between recommendation models and user-item interactions. Contextual bandit (CB) algorithms strive to make a good trade-off between exploration and exploitation so that users' potential interests have chances to expose. However, classical CB algorithms can only be applied to a small, sampled item set (usually hundreds), which forces the typical applications in recommender systems limited to candidate post-ranking, homepage top item ranking, ad creative selection, or online model selection (A/B test).In this paper, we introduce two simple but effective hierarchical CB algorithms to make a classical CB model (such as LinUCB and Thompson Sampling) capable to explore users' interest in the entire item space without limiting to a small item set. We first construct a hierarchy item tree via a bottom-up clustering algorithm to organize items in a coarse-to-fine manner. Then we propose a hierarchical CB (HCB) algorithm to explore users' interest on the hierarchy tree. HCB takes the exploration problem as a series of decisionmaking processes, where the goal is to find a path from the root to a leaf node, and the feedback will be back-propagated to all the nodes in the path. We further propose a progressive hierarchical CB (pHCB) algorithm, which progressively extends visible nodes which reach a confidence level for exploration, to avoid misleading actions on upper-level nodes in the sequential decision-making process. Extensive experiments on two public recommendation datasets demonstrate the effectiveness and flexibility of our methods.

show abstract

“…Hao et al [19], to solve a similar problem, employs clustering to group users in ad-hoc social network environments. Mahadik et al [32] deals with a scalable algorithm for the problem, DistCLUB. Gentile, Li and Zapella [14] investigate on-line clustering; the paper presents a strict analysis of the problem.…”

Section: Literature Reviewmentioning

confidence: 99%

Generalized Relational Tensors For Irregularly Sampled Time Series: Methods To Store, Re-generate, And Predict

Gromov

¹

,

Necheporenko

²

,

Gaisin

³

et al. 2021

Preprint

0

View full text Add to dashboard Cite

The paper deals with a generalized relational tensor, a novel discrete structure to store information about a time series, and algorithms (1) to fill the structure, (2) to generate a time series from the structure, and (3) to predict a time series, for both regularly and irregularly sampled time series. The algorithms combine the concept of generalized z-vectors with ant colony optimization techniques. In order to estimate quality of the storing/re-generating procedure, a difference between characteristics of the initial and regenerated time series is used. The structure allows working with a multivariate time series, with an irregularly sampled time series, and with a number of series as well. For chaotic time series, a difference between characteristics of the initial time series (the highest Lyapunov exponent, the auto-correlation function) and those of the time series re-generated from a structure is used to assess the effectiveness of the algorithms in question. The approach has shown fairly good results for periodic and benchmark chaotic time series and satisfactory results for real-world chaotic data.

show abstract

“…Moreover, HMAB aims to learn latent parameters for the nodes in the hierarchy tree, which is totally different from our goal of exploring users' latent interests. Distributed bandit algorithms, such as DCCB [14] and DistCLUB [20], aim to speed up the computation by distributing the workloads in parallel. However, these methods do not address the issue of searching from tremendous items, the computational cost is still too expensive for responding users' requests in an online manner (for example, how to response 100 users' concurrent requests within 10 milliseconds in a scenario involving one million items).…”

Section: Cluster-of-bandit Algorithmsmentioning

confidence: 99%

Show Me the Whole World: Towards Entire Item Space Exploration for Interactive Personalized Recommendations

Song¹,

Lian²,

Sun³

et al. 2021

Preprint

0

View full text Add to dashboard Cite

User interest exploration is an important and challenging topic in recommender systems, which alleviates the closed-loop effects between recommendation models and user-item interactions. Contextual bandit (CB) algorithms strive to make a good trade-off between exploration and exploitation so that users' potential interests have chances to expose. However, classical CB algorithms can only be applied to a small, sampled item set (usually hundreds), which forces the typical applications in recommender systems limited to candidate post-ranking, homepage top item ranking, ad creative selection, or online model selection (A/B test).In this paper, we introduce two simple but effective hierarchical CB algorithms to make a classical CB model (such as LinUCB and Thompson Sampling) capable to explore users' interest in the entire item space without limiting to a small item set. We first construct a hierarchy item tree via a bottom-up clustering algorithm to organize items in a coarse-to-fine manner. Then we propose a hierarchical CB (HCB) algorithm to explore users' interest on the hierarchy tree. HCB takes the exploration problem as a series of decisionmaking processes, where the goal is to find a path from the root to a leaf node, and the feedback will be back-propagated to all the nodes in the path. We further propose a progressive hierarchical CB (pHCB) algorithm, which progressively extends visible nodes which reach a confidence level for exploration, to avoid misleading actions on upper-level nodes in the sequential decision-making process.

show abstract

Fast distributed bandits for online recommendation systems

Cited by 52 publications

References 23 publications

Show Me the Whole World

Show Me the Whole World

Generalized Relational Tensors For Irregularly Sampled Time Series: Methods To Store, Re-generate, And Predict

Show Me the Whole World: Towards Entire Item Space Exploration for Interactive Personalized Recommendations

Contact Info

Product

Resources

About