Learning and adaptivity in interactive recommender systems

Abstract

Abstract. Recommender systems are intelligent applications that assist the users in a decision-making process by giving personalized product recommendations. Quite recently conversational approaches have been introduced to support a more interactive recommendation process. Notwithstanding the increased interactivity offered by these approaches, the system activity is rigid and follows an execution path that must be defined apriori, at design time. In this paper, we present a new type of recommender system capable of learning an interaction strategy by observing the effects of its actions on the user and on the final outcome of the recommendation session. We view the recommendation process as a sequential decision problem, and we model it as a Markov Decision Process. We evaluate this approach in a case study where the goal is to learn the best support strategy for advising a user in refining a query to a product catalogue. The experimental results demonstrate the value of our approach and show that an initial fixed strategy can be improved by first learning a model of the user behavior and then applying the Policy Iteration algorithm to compute the optimal recommendation policy.

Adaptive Recommender Systems for Travel Planning

“…The proposed model has been validated in a series of off-line experiments based on real-user interaction data acquired with the NutKing Recommender system . For lack of space we refer the reader to (Mahmood and Ricci, 2007a;Mahmood and Ricci, 2007b) for details about the results of such experiments.…”

“…Currently (November 2007), we have completed the system implementation phase, the off-line evaluation (Mahmood and Ricci, 2007a;Mahmood and Ricci, 2007b), the usability evaluation, and we are now going to perform the on-line system validation as described above with a travel planner system that we developed for Austria.info. The proposed model still requires further analysis for a number of reasons.…”

“…We tackle these requirements proposing a new type of Case-Based conversational recommender system that, rather than following a rigid interaction design, offers a range of information and decision support functions and, during the interaction, is able to improve an initial strategy and adopt an optimal one (Mahmood and Ricci, 2007a). The optimal strategy basically maximizes a reward function that models how much benefit the user gets from each interaction step.…”

“…In (Mahmood and Ricci, 2007a), we presented the more technical details of such a technology and we described the results of an evaluation, based on human/computer simulation. In that evaluation, when a search for a travel service (e.g.…”

Adapting the interaction state model in conversational recommender systems

“…This is an infeasible process in terms of budget, time, and task force. Hence, in a previous paper [6], we have tackled these requirements by proposing a new type of a recommender system which is able to autonomously improve a hard-coded strategy in order to learn and adopt an optimal one. Our system learns the optimal strategy through Reinforcement Learning (RL) techniques, basically by executing actions through trial-and-error, while interacting with the users.…”