Crafting the initial user experience to achieve community goals

Drenner, Sara; Sen, Shilad; Terveen, Loren

doi:10.1145/1454008.1454039

Cited by 29 publications

(32 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…According to behavioral decision theories [14], users are likely to settle on the immediate benefit of saving effort over the delayed gratification of higher accuracy. A number of works discussed in [28] support this principle (e.g., [12] [16]), which is also confirmed by a more recent online study presented in [13], which pinpoints that, in a content-based recommender, a higher perceived system effectiveness is related to reduced effort in the elicitation activity, measured in terms of amount of browsing before receiving recommendations.…”

Section: Related Workmentioning

confidence: 54%

“…The first assumption is that profiles length positively affects user utility. Some works show that profile length of new users is positively correlated to the accuracy of recommendations in term of user utility [12][13] [4]. However, this result cannot be easily generalized, as its supporting experiments are limited to item-based collaborative algorithms, and accuracy is measured only in terms of error metrics: RMSE [13] and MAE [4].…”

Section: Introductionmentioning

confidence: 99%

“…This preference elicitation process is fundamental both at cold-start time (i.e., when bootstrapping a new RS) and during the normal operational life of the system, and has effects along multiple dimensions. The preference elicitation strategy can affect the "new user utility" (how well the system can make good recommendations to the new user who is undergoing the elicitation process) and the "system or community utility" (how well the system can provide good recommendations to all users, given what it learns from the new user) [12][18] [27]. In addition, the elicitation process represents the user's initial experience with the recommender and is crucial to shape her attitude towards the system and her decision process and behavior (i.e., what she will do with the recommendations).…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

User effort vs. accuracy in rating-based elicitation

Cremonesi

Garzottto

Turrin³

2012

Proceedings of the Sixth ACM Conference on Recommender Systems

View full text Add to dashboard Cite

One of the unresolved issues when designing a recommender system is the number of ratings -i.e., the profile length -that should be collected from a new user before providing recommendations. A design tension exists, induced by two conflicting requirements. On the one hand, the system must collect "enough" ratings from the user in order to learn her/his preferences and improve the accuracy of recommendations. On the other hand, gathering more ratings adds a burden on the user, which may negatively affect the user experience. Our research investigates the effects of profile length from both a subjective (user-centric) point of view and an objective (accuracy-based) perspective. We carried on an offline simulation with three algorithms, and a set of online experiments involving overall 960 users and four recommender algorithms, to measure which of the two contrasting forces influenced by the number of collected ratings -recommendations relevance and burden of the rating process -has stronger effects on the perceived quality of the user experience. Moreover, our study identifies the potentially optimal profile length for an explicit, rating based, and human controlled elicitation strategy.

show abstract

Section: Related Workmentioning

confidence: 54%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

User effort vs. accuracy in rating-based elicitation

Cremonesi

Garzottto

Turrin³

2012

Proceedings of the Sixth ACM Conference on Recommender Systems

View full text Add to dashboard Cite

show abstract

“…This choice is motivated by three specific reasons: (i) we want to support users who have no rating history or who are not interested in logging into the system; (ii) we are interested in exploring a smooth integration of personalized recommendations in existing online booking systems; to enable explicit elicitation would require the introduction of an intrusive add-on; (iii) according to a large number of works, the lower effort of implicit elicitation (as compared to explicit elicitation) is related to higher perceived effectiveness of recommendations [9,10,14,23]. The implicit elicitation mechanism adopted in PoliVenus is the following: whenever a user interacts with an object on the interface, the system assigns a score to the hotel related to that object (e.g.…”

Section: Instrumentsmentioning

confidence: 99%

User-Centric vs. System-Centric Evaluation of Recommender Systems

Cremonesi

Garzotto

Turrin³

2013

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Abstract. Recommender Systems (RSs) aim at helping users search large amounts of contents and identify more effectively the items (products or services) that are likely to be more useful or attractive. The quality of a RS can be defined from two perspectives: system-centric, in which quality measures (e.g., precision, recall) are evaluated using vast datasets of preferences and opinions on items previously collected from users that are not interacting with the RS under study; user-centric, in which user measures are collected from users interacting with the RS under study. Prior research in e-commerce has provided some empirical evidence that system-centric and user-centric quality methods may lead to inconsistent results, e.g., RSs that were "best" according to systemcentric measures were not the top ones according to user-centric measures. The paper investigates if a similar mismatch also exists in the domain of e-tourism. We discuss two studies that have adopted a system-centric approach using data from 210000 users, and a user-centric approach involving 240 users interacting with an online hotel booking service. In both studies, we considered four RSs that employ an implicit user preference elicitation technique and different baseline and state-of-the-art recommendation algorithms. In these four experimental conditions, we compared system-centric quality measures against user-centric evaluation results. System-centric quality measures were consistent with usercentric measures, in contrast with past studies in e-commerce. This pinpoints that the relationship between the two kinds of metrics may depend on the business sector, is more complex that we may expect, and is a challenging issues that deserves further research.

show abstract

“…-broadcasting a message asking for specific contributions [1,5], -asking specific people to do specific tasks [1,4], -emphasizing uniqueness of the user's contributions [1,12], -providing social information and feedback [3,13], -assigning people to groups and setting group competitions [1,7], -setting personal or group goals [1,7,18], -reducing the effort required to identify tasks that are likely to be done by a user (i.e. recommend possible tasks that match the user's interests) [9,5].…”

Section: Related Workmentioning

confidence: 99%

Towards Adaptive Recruitment and Engagement Mechanisms in Social Systems

López

Brusilovsky

2012

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Abstract. Although successful online communities have engaged thousands of users, designers still struggle to recruit newcomers and increase current contribution rates. Related work on encouraging contributions has drawn from Social Psychology, Sociology and Economics theories. Engagement mechanisms embed the principles of these theories, and experimental studies evaluate the impact of different mechanisms on the contribution rates. Significant differences among alternative engagement mechanisms have been found, however, the results are sometimes contradictory for different groups of users. Our hypothesis is that the effectiveness of engagement mechanisms may depend on users' characteristics, and not solely on the mechanism itself. To start exploring this hypothesis, we performed a study to evaluate the impact of recruitment and engagement messages on different users' cohorts. Levels of current participation rates and demographic data were analyzed in order to explain differences in the impact of these engagement strategies.

show abstract

Crafting the initial user experience to achieve community goals

Cited by 29 publications

References 19 publications

User effort vs. accuracy in rating-based elicitation

User effort vs. accuracy in rating-based elicitation

User-Centric vs. System-Centric Evaluation of Recommender Systems

Towards Adaptive Recruitment and Engagement Mechanisms in Social Systems

Contact Info

Product

Resources

About