Attitudes towards user experience (UX) measurement

Abstract

User Experience (UX), as a recently established research area, is still haunted by the challenges of defining the scope of UX in general and operationalising experiential qualities in particular. To explore the basic question whether UX constructs are measurable, we conducted semi-structured interviews with ten UX researchers from academia and one UX practitioner from industry where a set of questions in relation to UX measurement were explored (Study 1). The interviewees expressed scepticism as well as ambivalence towards UX measures and shared anecdotes related to such measures in different contexts. Interestingly, the results suggested that design-oriented UX professionals tended to be sceptical about UX measurement.To examine whether such an attitude prevailed in the HCI community, we conducted a survey -UX Measurement Attitudes Survey (UXMAS) -with essentially the same set of 13 questions used in the interviews (Study 2). Specifically, participants were asked to rate a set of five statements to assess their attitude towards UX measurement, to identify (non)measurable experiential qualities with justifications, and to discuss the topic from the theoretical, methodological and practical perspective. The survey was implemented in a paper-based and an online format. Altogether, 367 responses were received; 170 of them were valid and analysed. The survey provided empirical evidence on this issue as a baseline for progress in UX measurement. Overall, the survey results indicated that the attitude towards UX measurement was more positive than that identified in the interviews, and there were nuanced views on details of UX measurement. Implications for enhancing the acceptance of UX measures and the interplay between UX evaluation and system development are drawn: UX modelling grounded in theories to link experiential qualities with outcomes; the development of UX measurement tools with good measurement properties, and education within the HCI community to disseminate validated models, and measurement tools as well as their successful applications. Mutual recognition of the value of objective measures and subjective accounts of user experience can enhance the maturity of this area.

Paper or Pixel? Comparing Paper- and Tool-Based Participatory Design Approaches

“…The drawbacks of exclusively quantitative (or qualitative) approaches are increasingly recognized in the field of HCI (e.g., [18]). For instance, the total number of user comments cannot tell whether the comments address the content or the user interface design and interaction concept of the system evaluated.…”

Modelagem sistêmica e simulação: estratégia de gestão no Processo de Projeto para a Experiência do Usuário (UXD)

“…These constructs are reflected in a way to optimize the use of data and excel in an approach that follows the directions of Law et al (2014) noting that there is much more to be glimpsed in the midst of this significant amount of data coming from evaluations performed with users.…”

“…From this moment UX is not only seen by the deliverables that it offers, but as a strategic area of the organization. In view of the above, it is verified that several researchers have directed their efforts to consolidate the UX area in different aspects, be it in the UX design process (Hartson & Pyla, 2012;Unger & Chandler, 2012), in specific methodologies for UX (Brown, 2013;Gothelf, 2013), in methods for the evaluation of UX (Law et al, 2009;Law et al, 2014) or through maturity models for UX that better target organizations (Carraro, 2006). However, it is necessary to have a clearer representation of the common design of multidisciplinary teams in UXD.…”

Whose Experience Do We Care About? Analysis of the Fitness of Scrum and Kanban to User Experience

“…UX is still plagued with definitional and measurement issues (Law, Roto, Hassenzahl, Vermeeren, & Kort, 2009;Law, van Schaik, & Roto, 2014). Despite its crudeness, the standards definition, ISO 9241-210: 2010, which comes with three notes, is often referenced: A person's perceptions and responses resulting from the use and/or anticipated use of a product, system or service.…”