A Review of Trends and Techniques in Recommender Systems

Rahul, *; Dahiya, Hritik; Singh, Divyansh

doi:10.1109/iot-siu.2019.8777645

Cited by 2 publications

(3 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…First, we report results for the Direct Method (DM), a supervised learning baseline where AttentionXML is trained with a partial classification loss, using only the feedback from actions for each instance. The deterministic policy picks the top-k actions from the predicted value, akin to Prabhu et al (2020).…”

Section: Competing Methods and Experimental Settingsmentioning

confidence: 99%

“…Such challenges have motivated the development of eXtreme multi-label classification (XMC) and eXtreme Regression (XR) (Bhatia et al 2016) methods, which focus on computational scalability issues and target settings involving millions of labels. These methods have had real-world applications in domains such as e-commerce (Agrawal et al 2013) and dynamic search advertising (Prabhu et al 2018(Prabhu et al , 2020.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Learning from eXtreme Bandit Feedback

Lopez

Dhillon

Jordan

2021

AAAI

View full text Add to dashboard Cite

We study the problem of batch learning from bandit feedback in the setting of extremely large action spaces. Learning from extreme bandit feedback is ubiquitous in recommendation systems, in which billions of decisions are made over sets consisting of millions of choices in a single day, yielding massive observational data. In these large-scale real-world applications, supervised learning frameworks such as eXtreme Multi-label Classification (XMC) are widely used despite the fact that they incur significant biases due to the mismatch between bandit feedback and supervised labels. Such biases can be mitigated by importance sampling techniques, but these techniques suffer from impractical variance when dealing with a large number of actions. In this paper, we introduce a selective importance sampling estimator (sIS) that operates in a significantly more favorable bias-variance regime. The sIS estimator is obtained by performing importance sampling on the conditional expectation of the reward with respect to a small subset of actions for each instance (a form of Rao-Blackwellization). We employ this estimator in a novel algorithmic procedure---named Policy Optimization for eXtreme Models (POXM)---for learning from bandit feedback on XMC tasks. In POXM, the selected actions for the sIS estimator are the top-p actions of the logging policy, where p is adjusted from the data and is significantly smaller than the size of the action space. We use a supervised-to-bandit conversion on three XMC datasets to benchmark our POXM method against three competing methods: BanditNet, a previously applied partial matching pruning strategy, and a supervised learning baseline. Whereas BanditNet sometimes improves marginally over the logging policy, our experiments show that POXM systematically and significantly improves over all baselines.

show abstract

Section: Competing Methods and Experimental Settingsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Learning from eXtreme Bandit Feedback

Lopez

Dhillon

Jordan

2021

AAAI

View full text Add to dashboard Cite

show abstract

“…Of particular interest is the off-policy setting where the training data is provided by a logging policy, which differs from the learner's policy and differs from the optimal policy. Such problems arise in many real-world problems, including supply chains, online markets, and recommendation systems [2], where abundant data is available in a logged format but not in a classical supervised learning format.…”

Section: Introductionmentioning

confidence: 99%

Learning from eXtreme Bandit Feedback

Lopez¹,

Dhillon²,

Jordan³

2020

Preprint

View full text Add to dashboard Cite

We study the problem of batch learning from bandit feedback in the setting of extremely large action spaces. Learning from extreme bandit feedback is ubiquitous in recommendation systems, in which billions of decisions are made over sets consisting of millions of choices in a single day, yielding massive observational data. In these large-scale real-world applications, supervised learning frameworks such as eXtreme Multi-label Classification (XMC) are widely used despite the fact that they incur significant biases due to the mismatch between bandit feedback and supervised labels. Such biases can be mitigated by importance sampling techniques, but these techniques suffer from impractical variance when dealing with a large number of actions. In this paper, we introduce a selective importance sampling estimator (sIS) that operates in a significantly more favorable biasvariance regime. The sIS estimator is obtained by performing importance sampling on the conditional expectation of the reward with respect to a small subset of actions for each instance (a form of Rao-Blackwellization). We employ this estimator in a novel algorithmic procedure-named Policy Optimization for eXtreme Models (POXM)-for learning from bandit feedback on XMC tasks. In POXM, the selected actions for the sIS estimator are the top-p actions of the logging policy, where p is adjusted from the data and is significantly smaller than the size of the action space. We use a supervised-to-bandit conversion on three XMC datasets to benchmark our POXM method against three competing methods: BanditNet, a previously applied partial matching pruning strategy, and a supervised learning baseline. Whereas BanditNet sometimes improves marginally over the logging policy, our experiments show that POXM systematically and significantly improves over all baselines.

show abstract

A Review of Trends and Techniques in Recommender Systems

Cited by 2 publications

References 26 publications

Learning from eXtreme Bandit Feedback

Learning from eXtreme Bandit Feedback

Learning from eXtreme Bandit Feedback

Contact Info

Product

Resources

About