State of the Art on the Cognitive Walkthrough Method, Its Variants and Evolutions

Abstract

This article discusses interactive system evaluation from the perspective of inspection methods, specifically the Cognitive Walkthrough (CW) method. The basic principles of CW are reviewed as proposed in the original version and the first two revisions. Then 11 significant extensions of CW are examined: Heuristic Walkthrough, The Norman Cognitive Walkthrough Method, Streamlined Cognitive Walkthrough, Cognitive Walkthrough for the Web, Groupware Walkthrough, Activity Walkthrough, Interaction Walkthrough, Cognitive Walkthrough with Users, Extended Cognitive Walkthrough, Distributed Cognitive Walkthrough, and Enhanced Cognitive Walkthrough. Four summaries are proposed:The first one concerns the conceptual, methododological, and technological aspects; the next two summaries deal with existing studies, first comparative and then noncomparative; and the last summary provides help for choosing a version or variant.

GenderMag: A Method for Evaluating Software's Gender Inclusiveness

“…It has been suggested this might be due to its structured nature [Hertzum and Jacobsen 1999]. Another strength of the CW method is that it can uncover design errors that may impede novices' learning by doing, but it can also uncover usability issues that extend beyond ease of learning [Mahatody et al 2010;Wharton et al 1994]. This strength has been attributed to its unconstrained nature [Hertzum and Jacobsen 1999] and correlation of ease of learning with ease of use and functionality [Mahatody et al 2010, Wharton et al 1994.…”

“…This version, which is often cited and applied today, did not require the evaluator to place as much emphasis on understanding the user's explicit and implicit goal structures for particular walkthrough steps. [Mahatody et al 2010]. There is also a Programming Walkthrough variant especially for evaluating programming environments [Bell et al 1991].…”

“…The CW focuses specifically on ease of learning [Blandford et al 2008;Lewis et al 1990;Mahatody et al 2010;Wharton et al 1994] and supports systematic evaluation of how a first-time user would carry out a task by using interface features. In a CW, a team of evaluators "walks through" the interface step by step, evaluating the interface's usability and learnability at each step, in the sequence a user would do when completing some particular task for the first time.…”

Formative Evaluation of IT-based Services: A Case Study of a Meal Planning Service

“…UEMs can be categorized in different ways, but there are essentially three main types of methods : inspection (or expert), evaluation (e.g. heuristic evaluation, Hvannberg et al, 2007;Nielsen, 1992 and cognitive walkthrough Chikhaoui and Pigot, 2010;Langdon et al, 2010;Lewis et al, 1990;Mahatody et al, 2010), model evaluation Sutcliffe et al, 2000) and user evaluation (e.g. usability testing Nørgaard and Hornbaek, 2006).…”

A Design Science Approach to Interactive Greenhouse Climate Control Using Lego Mindstorms for Sensor-Intensive Prototyping

“…Prototype 6 was a high fidelity prototype of a part of the to-be-designed climate management system, the "side-bar" which gave decision support to the grower based on model simulations of the climate ( Figure 13). Three student groups from a second year computer science and business administration study from the second author's course conducted a cognitive walkthrough [26] of the prototype in Figure 13. The results of the cognitive walkthrough were sent to the developers, who stated that some results -that is, some design suggestions in reference to consistency, level of detail, and scalability of graphs were particularly useful in the further development of the system.…”