User-centric evaluation of a K-furthest neighbor collaborative filtering recommender algorithm

Said, Alan; Fields, Ben; Jain, Brijnesh J.; Albayrak, Şahin

doi:10.1145/2441776.2441933

Cited by 89 publications

(51 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Based on the evaluation outcomes, the user-centric evaluation results in this study certainly do not confirm the traditional offline evaluation results. The few other recommender system studies that also take a user-centric view, show a similar inconsistency, although different in their details, between traditional evaluations and user-centric evaluations [8], [29], [30].…”

Section: Discussionmentioning

confidence: 97%

“…However, making use of the whole framework can be very time consuming for participants since it includes many metrics. Therefore, we only focus on five important metrics that have been identified in the literature on recommender systems user studies as indicators of users satisfaction on the recommendations made for them [3], [29], [30]. By tending towards simplicity we seek to guarantee responsiveness.…”

Section: Questionnairementioning

confidence: 99%

See 1 more Smart Citation

User-Centric Evaluation of Recommender Systems in Social Learning Platforms: Accuracy is Just the Tip of the Iceberg

Fazeli

Drachsler

Bitter-Rijpkema

et al. 2018

IEEE Trans. Learning Technol.

View full text Add to dashboard Cite

Abstract-Recommender systems provide users with content they might be interested in. Conventionally, recommender systems are evaluated mostly by using prediction accuracy metrics only. But the ultimate goal of a recommender system is to increase user satisfaction. Therefore, evaluations that measure user satisfaction should be also performed before deploying a recommender system to a real target environment. Such evaluations are laborious and complicated compared to the traditional, data-centric evaluations, though. In this study, we investigate the added value of user-centric evaluations and how user satisfaction of a recommender system is related to its performance in terms of accuracy metrics. We conduct both a data-centric evaluation and a user-centric evaluation on the same data collected from an authentic social learning platform. Our findings suggest that user-centric evaluation results are not necessarily in line with data-centric evaluation results. We conclude that the traditional evaluation of recommender systems in terms of prediction accuracy does not suffice to judge performance of recommender systems on the user side. Moreover, the user-centric evaluation provides valuable insights on how candidate algorithms perform on each of the five quality metrics: usefulness, accuracy, novelty, diversity, and serendipity of the recommendations.

show abstract

Section: Discussionmentioning

confidence: 97%

Section: Questionnairementioning

confidence: 99%

User-Centric Evaluation of Recommender Systems in Social Learning Platforms: Accuracy is Just the Tip of the Iceberg

Fazeli

Drachsler

Bitter-Rijpkema

et al. 2018

IEEE Trans. Learning Technol.

View full text Add to dashboard Cite

show abstract

“…Many user study results indicated that overall satisfaction with the beyond-accuracy aspect adopted recommenders is higher than that with the accuracy-focused recommenders [67], [70], [84], even in cases where the accuracy of the beyond-accuracy adopted recommenders is lower than that of competitors [10], [30], [61], [85], [96]. However, it is important to strike an appropriate balance between accuracy and beyond-accuracy.…”

Section: Resultsmentioning

confidence: 99%

“…However, there exist studies whose authors took different strategies. Said et al [61] proposed a recommender that recommends the items with which the furthest neighbors infrequently interact. The furthest neighbors indicate the group of users whose purchase pattern is dissimilar to a target.…”

Section: Neighbors/models With Multi-aspectsmentioning

confidence: 99%

“…Unlike with off-line employment, on-line volunteers and cloud-sourced workers perform evaluation tasks in remote places, far from where the conductor is. Therefore, for these types of studies, it is better to prepare a mechanism that guarantees evaluation quality [61]. Questionnaire: All of the 15 studies to feature small user samples had used a questionnaire as an evaluation methodlikely on account of sample size.…”

Section: Small-sized User Studiesmentioning

confidence: 99%

See 1 more Smart Citation

A Survey on Recommendation Methods Beyond Accuracy

Han

Yamana

2017

IEICE Trans. Inf. & Syst.

View full text Add to dashboard Cite

SUMMARYIn recommending to another individual an item that one loves, accuracy is important, however in most cases, focusing only on accuracy generates less satisfactory recommendations. Studies have repeatedly pointed out that aspects that go beyond accuracy-such as the diversity and novelty of the recommended items-are as important as accuracy in making a satisfactory recommendation. Despite their importance, there is no global consensus about definitions and evaluations regarding beyondaccuracy aspects, as such aspects closely relate to the subjective sensibility of user satisfaction. In addition, devising algorithms for this purpose is difficult, because algorithms concurrently pursue the aspects in trade-off relation (i.e., accuracy vs. novelty). In the aforementioned situation, for researchers initiating a study in this domain, it is important to obtain a systematically integrated view of the domain. This paper reports the results of a survey of about 70 studies published over the last 15 years, each of which addresses recommendations that consider beyond-accuracy aspects. From this survey, we identify diversity, novelty, and coverage as important aspects in achieving serendipity and popularity unbiasedness-factors that are important to user satisfaction and business profits, respectively. The five major groups of algorithms that tackle the beyond-accuracy aspects are multi-objective, modified collaborative filtering (CF), clustering, graph, and hybrid; we then classify and describe algorithms as per this typology. The off-line evaluation metrics and user studies carried out by the studies are also described. Based on the survey results, we assert that there is a lot of room for research in the domain. Especially, personalization and generalization are considered important issues that should be addressed in future research (e.g., automatic per-user-trade-off among the aspects, and properly establishing beyond-accuracy aspects for various types of applications or algorithms).

show abstract

Serendipity in the city: User evaluations of urban recommender systems

Smets

Vannieuwenhuyze

Ballon

2021

Asso for Info Science & Tech

View full text Add to dashboard Cite

The contemporary city is increasingly being labeled as a smart city consisting of both physical and virtual spaces. This digital augmentation of urban life sets the scene for urban recommender systems to help citizens dealing with the abundance of digital information and corresponding choice overload, for example, by recommending the best place to have dinner based on your personal profile. There are, however, concerns that this kind of algorithmic filtering could lead to homogenization of urban experiences and a decline of social cohesion among citizens. To overcome this issue, scholars increasingly encourage the introduction of serendipity in all types of recommender systems. Nonetheless, it remains unclear how this can be achieved in practice. In this work, we study user evaluations of serendipity in urban recommender systems through a survey among 1641 citizens. More specifically, we study which characteristics of recommended items contribute to serendipitous experiences and to what extent this increases user satisfaction and conversion. Our results align with findings in other application domains in the sense that there is a strong relation between the relevance and novelty of recommendations and the corresponding experienced serendipity. Moreover, serendipitous recommendations are found to increase the chance of users following up on these recommendations.

show abstract

User-centric evaluation of a K-furthest neighbor collaborative filtering recommender algorithm

Cited by 89 publications

References 21 publications

User-Centric Evaluation of Recommender Systems in Social Learning Platforms: Accuracy is Just the Tip of the Iceberg

User-Centric Evaluation of Recommender Systems in Social Learning Platforms: Accuracy is Just the Tip of the Iceberg

A Survey on Recommendation Methods Beyond Accuracy

Serendipity in the city: User evaluations of urban recommender systems

Contact Info

Product

Resources

About