Despite the growing interest in user experience (UX), it has been hard to gain a common agreement on the nature and scope of UX. In this paper, we report a survey that gathered the views on UX of 275 researchers and practitioners from academia and industry. Most respondents agree that UX is dynamic, context-dependent, and subjective. With respect to the more controversial issues, the authors propose to delineate UX as something individual (instead of social) that emerges from interacting with a product, system, service or an object. The draft ISO definition on UX seems to be in line with the survey findings, although the issues of experiencing anticipated use and the object of UX will require further explication. The outcome of this survey lays ground for understanding, scoping, and defining the concept of user experience.

User Experience (UX), as a recently established research area, is still haunted by the challenges of defining the scope of UX in general and operationalising experiential qualities in particular. To explore the basic question whether UX constructs are measurable, we conducted semi-structured interviews with ten UX researchers from academia and one UX practitioner from industry where a set of questions in relation to UX measurement were explored (Study 1). The interviewees expressed scepticism as well as ambivalence towards UX measures and shared anecdotes related to such measures in different contexts. Interestingly, the results suggested that design-oriented UX professionals tended to be sceptical about UX measurement.To examine whether such an attitude prevailed in the HCI community, we conducted a survey -UX Measurement Attitudes Survey (UXMAS) -with essentially the same set of 13 questions used in the interviews (Study 2). Specifically, participants were asked to rate a set of five statements to assess their attitude towards UX measurement, to identify (non)measurable experiential qualities with justifications, and to discuss the topic from the theoretical, methodological and practical perspective. The survey was implemented in a paper-based and an online format. Altogether, 367 responses were received; 170 of them were valid and analysed. The survey provided empirical evidence on this issue as a baseline for progress in UX measurement. Overall, the survey results indicated that the attitude towards UX measurement was more positive than that identified in the interviews, and there were nuanced views on details of UX measurement. Implications for enhancing the acceptance of UX measures and the interplay between UX evaluation and system development are drawn: UX modelling grounded in theories to link experiential qualities with outcomes; the development of UX measurement tools with good measurement properties, and education within the HCI community to disseminate validated models, and measurement tools as well as their successful applications. Mutual recognition of the value of objective measures and subjective accounts of user experience can enhance the maturity of this area.

Understanding the relation between usability measures seems crucial to deepen our conception of usability and to select the right measures for usability studies. We present a meta-analysis of correlations among usability measures calculated from the raw data of 73 studies. Correlations are generally low: effectiveness measures (e.g., errors) and efficiency measures (e.g., time) have a correlation of .247 ± .059 (Pearson's product-moment correlation with 95% confidence interval), efficiency and satisfaction (e.g., preference) one of .196 ± .064, and effectiveness and satisfaction one of .164 ± .062. Changes in task complexity do not influence these correlations, but use of more complex measures attenuates them. Standard questionnaires for measuring satisfaction appear more reliable than homegrown ones. Measures of users' perceptions of phenomena are generally not correlated with objective measures of the phenomena. Implications for how to measure usability are drawn and common models of usability are criticized.

No abstract