Obstacles to usability evaluation in practice

Bak, Jakob; Nguyen, Kim Anh; Risgaard, Peter; Stage, Jan

doi:10.1145/1463160.1463164

Cited by 58 publications

(16 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…While previous empirical studies report that usability is not understood properly, especially among technical stakeholders (e.g. [25] (2008)) we observed that in the studied organizations practitioners have fair knowledge and awareness about usability. This can be an indication that industrial knowledge and awareness of usability is improving.…”

Section: Discussioncontrasting

confidence: 67%

See 1 more Smart Citation

Peer Review #1 of "Integrating User eXperience practices into software development processes: implications of the UX characteristics (v0.1)"

2017

View full text Add to dashboard Cite

User eXperience (UX) is a key factor in the success of software systems. Many software companies face challenges in their work with UX. Existing research does not analyze UX practices and challenges in relation to other software quality characteristics or, in particular, in relation to usability. A better understanding of these challenges can help researchers and practitioners better address them in the future. In this empirical study, we have interviewed 17 practitioners with different backgrounds and occupations from eight software development companies. Their responses are coded, and analyzed with thematic analysis. We report 8 themes of challenges that practitioners face in their work with UX. While some of these challenges partly overlap with those reported in existing literature about usability or other software quality characteristics, the participants of our study either view many of the challenges as unique to UX, or more severe in the case of UX. Although at a superficial level challenges of UX and other quality characteristics overlap, we differentiate these challenges at a deeper level through the main characteristics of UX: subjective, holistic, dynamic, context-dependent and worthwhile. In particular, we identified that these characteristics have at least 20 implications (i.e. additional difficulties) for day-today work of practitioners. We found that 11 of these implications have been previously reported in literature. However, to the best of our knowledge, the remaining nine implications are unique to our study. These implications can explain why practitioners perceive the challenges to be more severe than for other quality characteristics. Most importantly, they can explain the industry's lopsided focus on the pragmatic aspect of UX. Our findings can be useful for researchers in identifying new and industry-relevant research areas and for practitioners to learn from empirically investigated challenges in UX work, and base their improvement efforts on such knowledge. Identifying and investigating the overlaps underlines the importance of these challenges, and can also help finding research areas not only for enhancing UX work but also software quality in general. It also makes it easier for practitioners to spot, better understand as well as find mitigation strategies for UX, through learning from past experiences and developments in the area of software quality.

show abstract

Section: Discussioncontrasting

confidence: 67%

“…Bak et al [25] report that developers' minds are set more on the programming aspect of the product than its usability, and they often misunderstand the term "usability evaluation" and often relate it to functionality.…”

Section: Background and Related Workmentioning

confidence: 99%

Peer Review #1 of "Integrating User eXperience practices into software development processes: implications of the UX characteristics (v0.1)"

2017

View full text Add to dashboard Cite

show abstract

“… Organizing and conducting user tests is relatively expensive because it requires a high demand for human and time resources [8], [9];  Small software companies do not have the funds to pay for complete consultancy or for involving usability specialists, as they are expensive to hire [10];  Difficulty of getting potential users to participate in usability evaluations [9], [11];  It is not always possible to increase the coverage of evaluated features as evaluating every single aspect of UI [8] may itself also not be possible. Herein we focus only on automated usability evaluation.…”

Section: Introductionmentioning

confidence: 99%

Design-Time Web Usability Evaluation with Guideliner

Marenkov

Robal

Kalja

2018

CSIMQ

View full text Add to dashboard Cite

Abstract. The diversity of smartphones and tablet computers has become an integral part of modern life. An essential requirement for web application development is following web usability guidelines, while designing web user interface (UI). Even a minor change in UI could lead to usability problems. Empirical evaluation methods like interviews and questionnaires with user-tests and card sorting are effective in finding such problems. Nevertheless, there are multiple obstacles preventing the application of these methods especially for evaluating minor UI changes, for instance, due to the time and human-resources they require, and the amount of data to be processed. The purpose of this current publication is to present Guideliner -a tool for implementation-time automatic evaluation of web UI conformance to predefined usability guidelines. The main contribution of the presented solution is enabling immediate cost-efficient and automated web UI evaluation that conforms to available and set standards.

show abstract

“…In recent years, an important number of projects (Agarwal & Meyer, 2009;Lera & Garreta-domingo, 2007;Branco, Firth, Encarnação & Bonato, 2005;Petrie, 2010)we explored the use of two techniques: an emotional think aloud protocol and an emotion word prompt list (EWPL have aimed to address the emotional aspects in the evaluation of usability of interactive systems (and to enhance user experience). Objective and subjective aspects of emotions are addressed by various methods used to evaluate interactive systems (Bak, Nguyen, Risgaard & Stage, 2008) Still, existing methods have significant limitations. Moreover, in spite of the diversity of approaches, a welldefined methodology for incorporating emotions in the process of evaluating usability and user experience remains to be developed.…”

Section: Introductionmentioning

confidence: 99%

Evaluating interactive systems from an emotional perspective

Collázos

Saltiveri

Huitr

2014

Rev. Guillermo Ockham

View full text Add to dashboard Cite

<p>In this paper, we introduce a model that aims to provide guidelines that will strengthen the evaluation of interactive systems by assisting in the identification and analysis of emotions evoked by users during system usage. We briefly discuss related projects that have included emotions in the evaluation of interactive systems. The research presented here is a preliminary work towards the inclusion of emotions during the evaluation of interactive systems. Our model is presented including details of each of its phases. We discussed preliminary results of the application of the model to evaluating a Virtual Learning Environment. Our approach comprises four major phases: Selection of relevant emotions; analysis of relationships between emotions and interactive systems; selection of detection mechanisms; and application of evaluation methods.</p>

show abstract

Obstacles to usability evaluation in practice

Cited by 58 publications

References 21 publications

Peer Review #1 of "Integrating User eXperience practices into software development processes: implications of the UX characteristics (v0.1)"

Peer Review #1 of "Integrating User eXperience practices into software development processes: implications of the UX characteristics (v0.1)"

Design-Time Web Usability Evaluation with Guideliner

Evaluating interactive systems from an emotional perspective

Contact Info

Product

Resources

About