Evaluating conversational recommender systems

Jannach, Dietmar

doi:10.1007/s10462-022-10229-x

Cited by 18 publications

(6 citation statements)

References 106 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In the past few years, chatbots have been adopted to support online customers' services, in users' assistants (e.g., Alexa (Hardesty 2022), or Siri (Siri Team 2017)), in automotive voice assistant (Capgemini 2019) and many more. Chatbots have also been exploited in recommendation systems, to help users to interact more naturally while allowing companies to reduce human intervention (Jannach 2023). Tourism operators have been using chatbot-based recommender systems for customer service, e.g., to give suggestions for travel or hospitality services, but also to support decision about where to go on holiday and what to do during it (Camilleri and Troise 2023).…”

Section: Chatgpt: Technological Capabilities and Applicationsmentioning

confidence: 99%

ChatGPT for e-Tourism: a technological perspective

Mich

Garigliano²

2023

Inf Technol Tourism

View full text Add to dashboard Cite

The interest around the conversational system ChatGPT has exploded since OpenAI launched the GPT-3 prototype at the end of November 2022. Applications of ChatGPT could be disruptive for the digital transformation processes, challenging societies, companies, and people in a possibly unprecedented way. Alongside remarkable performances, there are risks related to its intrinsic limitations. In this viewpoint paper we introduce ChatGPT, highlighting its main characteristics, with the aim of offering some general and preliminary guidelines for its adoption in the tourism field.

show abstract

Section: Chatgpt: Technological Capabilities and Applicationsmentioning

confidence: 99%

ChatGPT for e-Tourism: a technological perspective

Mich

Garigliano²

2023

Inf Technol Tourism

View full text Add to dashboard Cite

show abstract

“…We chose this age range because at this age, users are considered to be familiar with digital environments such as chatbots and are still enthusiastic about reading books. There are 7 statements in the questionnaire, grouped into 6 factors, namely informative (INF), easy to use (ETU), perceived recommendation quality (PRQ), ease of understanding (EOU), trust (TR), and perceived efficiency (PE) [24]. the calculation to get the final score for each statement in the questionnaire is shown.…”

Section: User Satisfactionmentioning

confidence: 99%

Chatbot-Based Book Recommender System Using Singular Value Decomposition

Attalariq,

Baizal

2023

josh

View full text Add to dashboard Cite

In the era of information overload, finding the right book that matches one's preferences and interests has become a challenging task for users as many online book provider service websites such as Amazon, Goodreads, and Gramedia provide books of various types and choices. Recommender systems can be used in addressing such issues, it works by filtering information that provides predictions and suggests the best product or service to the user. Currently, various book recommender systems have been developed, but the systems do not provide interaction between the user and the system. Therefore, we propose a recommender system built with a conversational approach so that it can interact with natural language. Recommender system built using matrix factorization method with Singular Value Decomposition (SVD) algorithm, SVD is proven to have advantages for handling large datasets, extracting features, reducing noise and dimensionality so as to speed up computation. We performed two types of evaluation on the system. First, we tested the prediction accuracy using Root Mean Squared Error (RMSE) and Mean Absolute Error (MAE) metrics. Second, we use questionnaires to measure user satisfaction levels. The evaluation of the system shows that the results of the prediction accuracy obtain an MAE value of 0.6481 and an RMSE value of 0.8287. Then, the accuracy performance of the system found that 83.2% of users get recommendations according to their interests. The user satisfaction with the whole system is 87.9%. The system built can provide a fairly good recommendation performance, and the chatbot can interact well with users based on the evaluation results obtained.

show abstract

“…Evaluating the CRS is a multitude problem as it requires various approaches to consider aiming at various dimensions, see for example recent surveys on the evaluation of CRS [34,35]. Computational experiments, where instead of human judges, researchers often rely on objective ("offline") metrics in order to assess the quality of a system.…”

Section: Crs Evaluationmentioning

confidence: 99%

INSPIRED2: An Improved Dataset for Sociable Conversational Recommendation

Manzoor¹,

Jannach²

2022

Preprint

Self Cite

View full text Add to dashboard Cite

Conversational recommender systems (CRS) that interact with users in natural language utilize recommendation dialogs collected with the help of paired humans, where one plays the role of a seeker and the other as a recommender. These recommendation dialogs include items and entities to disclose seekers' preferences in natural language. However, in order to precisely model the seekers' preferences and respond consistently, mainly CRS rely on explicitly annotated items and entities that appear in the dialog, and usually leverage the domain knowledge. In this work, we investigate INSPIRED, a dataset consisting of recommendation dialogs for the sociable conversational recommendation, where items and entities were explicitly annotated using automatic keyword or pattern matching techniques. To this end, we found a large number of cases where items and entities were either wrongly annotated or missing annotations at all. The question however remains to what extent automatic techniques for annotations are effective. Moreover, it is unclear what is the relative impact of poor and improved annotations on the overall effectiveness of a CRS in terms of the consistency and quality of responses. In this regard, first, we manually fixed the annotations and removed the noise in the INSPIRED dataset. Second, we evaluate the performance of several benchmark CRS using both versions of the dataset. Our analyses suggest that with the improved version of the dataset, i.e., INSPIRED2, various benchmark CRS outperformed and that dialogs are rich in knowledge concepts compared to when the original version is used. We release our improved dataset (INSPIRED2) publicly at https://github.com/ahtsham58/INSPIRED2.

show abstract

Evaluating conversational recommender systems

Cited by 18 publications

References 106 publications

ChatGPT for e-Tourism: a technological perspective

ChatGPT for e-Tourism: a technological perspective

Chatbot-Based Book Recommender System Using Singular Value Decomposition

INSPIRED2: An Improved Dataset for Sociable Conversational Recommendation

Contact Info

Product

Resources

About