Back to the Manifold: Recovering from Out-of-Distribution States

Reichlin, Alfredo; Marchetti, Giovanni Luca; Yin, H.; Ghadirzadeh, Ali; Kragić, Danica

doi:10.48550/arxiv.2207.08673

Cited by 1 publication

(1 citation statement)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Additionally, acknowledging the presence of uncertainty in the deployment of RL-based recommender systems paves the way towards solutions that are robust or resilient to such uncertainty. For instance, Oosterhuis and de Rijke [2021] propose a criterion for fallback to a safer policy when out-of-distribution (although in a different context, i.e., counterfactual learning to rank), and Ghosh et al [2022]; Reichlin et al [2022] propose adaptive offline RL policies that are able to recover from stepping in uncertain states during deployment by branching back to supported states. We hope that future research in recommender systems will put stronger emphasis on these aspects and reduce the gap between offline and online performance.…”

Section: Uncertainty-aware Evaluationmentioning

confidence: 99%

Offline Evaluation for Reinforcement Learning-Based Recommendation

Deffayet

Thonet²,

Renders³

et al. 2022

SIGIR Forum

View full text Add to dashboard Cite

In this paper, we argue that the paradigm commonly adopted for offline evaluation of sequential recommender systems is unsuitable for evaluating reinforcement learning-based recommenders. We find that most of the existing offline evaluation practices for reinforcement learning-based recommendation are based on a next-item prediction protocol, and detail three shortcomings of such an evaluation protocol. Notably, it cannot reflect the potential benefits that reinforcement learning (RL) is expected to bring while it hides critical deficiencies of certain offline RL agents. Our suggestions for alternative ways to evaluate RL-based recommender systems aim to shed light on the existing possibilities and inspire future research on reliable evaluation protocols.

show abstract

Section: Uncertainty-aware Evaluationmentioning

confidence: 99%