Early Findings from a Large-Scale User Study of CHESTNUT: Validations and Implications

Abstract

Towards a serendipitous recommender system with user-centred understanding, we have built CHESTNUT , an Information Theorybased Movie Recommender System, which introduced a more comprehensive understanding of the concept. Although off-line evaluations have already demonstrated that CHESTNUT has greatly improved serendipity performance, feedback on CHESTNUT from real-world users through online services are still unclear now. In order to evaluate how serendipitous results could be delivered by CHESTNUT , we consequently designed, organized and conducted large-scale user study, which involved 104 participants from 10 campuses in 3 countries. Our preliminary feedback has shown that, compared with mainstream collaborative filtering techniques, though CHESTNUT limited users' feelings of unexpectedness to some extent, it showed significant improvement in their feelings about certain metrics being both beneficial and interesting, which substantially increased users' experience of serendipity. Based on them, we have summarized three key takeaways, which would be beneficial for further designs and engineering of serendipitous recommender systems, from our perspective.