Two-Stage Neural Contextual Bandits for Personalised News Recommendation

Abstract

We consider the problem of personalised news recommendation where each user consumes news in a sequential fashion. Existing personalised news recommendation methods focus on exploiting user interests and ignores exploration in recommendation, which leads to biased feedback loops and hurt recommendation quality in the long term. We build on contextual bandits recommendation strategies which naturally address the exploitation-exploration trade-off. The main challenges are the computational efficiency for exploring the large-scale item space and utilising the deep representations with uncertainty. We propose a two-stage hierarchical topic-news deep contextual bandits framework to efficiently learn user preferences when there are many news items. We use deep learning representations for users and news, and generalise the neural upper confidence bound (UCB) policies to generalised additive UCB and bilinear UCB. Empirical results on a large-scale news recommendation dataset show that our proposed policies are efficient and outperform the baseline bandit policies.

On Instance-Dependent Bounds for Offline Reinforcement Learning with Linear Function Approximation

“…It provides a practical setting where logged datasets are abundant but exploring the environment can be costly due to computational, economic, or ethical reasons. It finds applications in a number of important domains including healthcare (Gottesman et al 2019;Nie, Brunskill, and Wager 2021), recommendation systems (Strehl et al 2010;Thomas et al 2017;Zhang et al 2022a), econometrics (Kitagawa and Tetenov 2018; Athey and Wager 2021), and more.…”