IntroductionInvolving users in usability evaluation is valuable when designing information and communication technology (ICT), and a range of usability evaluation methods (UEM) support user involvement. Relevant methods include adaptations of usability testing [1], usability inspection methods such as pluralistic walkthrough [2], and inquiry methods such as interviews [3], and focus groups [4].Users involved in usability evaluation may generate two types of data. We term these interaction data and design feedback. Interaction data are recordings of the actual use of an interactive system, such as observational data, system logs, and data from think-aloud protocols. Design feedback are data on users' reflections concerning an interactive system, such as comments on experiential issues, considerations of the system's suitability for its context of use, usability problem predictions, and design suggestions.
AbstractAs part of usability evaluation, users may be invited to offer their reflections on the system being evaluated. Such reflections may concern the system's suitability for its context of use, usability problem predictions, and design suggestions. We term the data resulting from such reflections users' design feedback. Gathering users' design feedback as part of usability evaluation may be seen as controversial, and the current knowledge on users' design feedback is fragmented. To mitigate this, we have conducted a literature review. The review provides an overview of the benefits and limitations of users' design feedback in usability evaluations. Following an extensive search process, 31 research papers were identified as relevant and analysed. Users' design feedback is gathered for a number of distinct purposes: to support budget approaches to usability testing, to expand on interaction data from usability testing, to provide insight into usability problems in users' everyday context, and to benefit from users' knowledge and creativity. Evaluation findings based on users' design feedback can be qualitatively different from, and hence complement, findings based on other types of evaluation data. Furthermore, findings based on users' design feedback can hold acceptable validity, though the thoroughness of such findings may be questioned. Finally, findings from users' design feedback may have substantial impact in the downstream development process. Four practical implications are highlighted, and three directions for future research are suggested.
RESEARCHFølstad Hum. Cent. Comput. Inf. Sci. (2017) Cent. Comput. Inf. Sci. (2017) 7:19 The value of interaction data in evaluation is unchallenged. Interaction data is held to be a key source of insight in the usability of interactive systems and has been the object of thorough scientific research. Numerous empirical studies concern the identification of usability problems on the basis of observable user behaviour [5]. Indeed, empirical UEM assessments are typically done by comparing the set of usability problems identified through the assessed UEM with a ...