Better discount evaluation: illustrating how critical parameters support heuristic creation

Abstract

This paper describes a heuristic creation process based on the notion of critical parameters, and a comparison experiment that demonstrates the utility of heuristics created for a specific system class. We focus on two examples of using the newly created heuristics to illustrate the utility of the usability evaluation method, as well as to provide support for the creation process, and we report on successes and frustrations of two classes of users, novice evaluators and domain experts, who identified usability problems with the new heuristics. We argue that establishing critical parameters for other domains will support efforts in creating tailored evaluation tools. q

Heuristic evaluation: Comparing ways of finding and reporting usability problems

“…Indeed, there have been some recent studies in developing tailor-made usability heuristics to fit special application contexts (e.g. ambient displays, (Mankoff et al, 2003)) and large screen display information exhibits (Somervell et al, 2003), and they are proven to be more effective than Nielsen's heuristics (Somervell and McCrickard, 2005). A tool, with the added intelligence of creating taxonomies of problems in various application contexts to context specific heuristics, can make such a selection of heuristics more effective.…”

“…Furthermore, with the high complexity of an application but limited resource, evaluation thus needs to be scoped and goal-oriented. Somervell's and McCrickard's (2005) work focuses on the creation of so-called 'critical parameter-based' heuristics, based on critical parameters proposed by Newman and Taylor (1999). For the class of large-scale display systems Somervell and McCrickard identified three critical parameters, namely Interruption, Reaction and Comprehension.…”

“…Clearly, the mapping of heuristics to the system's characteristics is important. But Somervell and McCrickard (2005) emphasize that mapping is not at the level of an individual system but a class of systems. This assertion aligns with Nielsen's recommendation ''Furthermore, it is possible to develop category-specific heuristics that apply to a specific class of products as a supplement to the general heuristics'' (Nielsen, 1994b, p. 29).…”

“…This assertion aligns with Nielsen's recommendation ''Furthermore, it is possible to develop category-specific heuristics that apply to a specific class of products as a supplement to the general heuristics'' (Nielsen, 1994b, p. 29). He further suggests performing competitive analysis and user testing to create abstract categories of specific heuristics, but Somervell and McCrickard (2005) use claims-analysis as a basis for heuristic creation. A registration tool may help collect the most commonly found faults in this class of applications or design features, depending on the granularity of the subject of evaluation.…”

Evidence Based Design of Heuristics for Computer Assisted Assessment

“…The raw count of the number of usability problems identified may not be an appropriate indicator of the effectiveness of a set of heuristics [7]. To validate heuristics, certain criteria are used including thoroughness [8], correctness, coverage and terminology [4]. Correctness refers to the terminology used in the specifications of the heuristics, and whether the descriptions provide sufficient information.…”