Heuristics for information visualization evaluation

Abstract

Heuristic evaluation is a well known discount evaluation technique in human-computer interaction (HCI) but has not been utilized in information visualization (InfoVis) to the same extent. While several sets of heuristics have been used or proposed for InfoVis, it is not yet known what kind of heuristics are useful for finding general InfoVis problems. We performed a meta-analysis with the goal of exploring the issues of heuristic evaluation for InfoVis. This meta-analysis concentrates on issues pertaining to the selection and organization of heuristics, and the process itself. For this purpose, we used three sets of previously published heuristics to assess a visual decision support system that is used to examine simulation data. The meta-analysis shows that the evaluation process and results have a high dependency on the heuristics and the types of evaluators chosen. We describe issues related to interpretation, redundancy, and conflict in heuristics. We also provide a discussion of generalizability and categorization of these heuristics.

Design Study Methodology: Reflections from the Trenches and the Stacks

“…The visualization literature contains a multitude of proposed methods for evaluating visualization tools in the wild [13,33,69,85,95], as well as guidance on using them [39,50]. The most common form of validation are case studies with real users, real problems, and real data, as featured in many strong design studies by others [35,58], and many of our own [10,40,43,44,45,46,71,74,76,77].…”

“…The relevant literature instead focuses on methods for designing [1,42,66,79,82,90,91] and evaluating [13,33,39,50,68,69,76,80,85,86,95] visualization tools. We distinguish between methods and methodology with the analogy of cooking; methods are like ingredients, whereas methodology is like a recipe.…”

User Evaluation Methods for Visual Web Search Interfaces

“…Many researchers have proposed the use of inspection methods as part of their evaluation methodology for information visualization systems [23,25,34,35,38]. In general, there are two styles of inspection method: those that are based on a set of design guidelines, such as heuristic evaluations; and those that are based on performing specific tasks, such as cognitive walkthroughs.…”

“…Both Freitas et al [8] and Zuk et al [38] have suggested that inspection methods can be extended to include the consideration of specific issues relevant to information visualization. Their extensions include evaluating the codification of information, the completeness of the information being visually represented, and the spatial organization of the objects in the visual display.…”

Visualização de dados para a promoção da experiência histórica: experiência e/ou individuação no processo de aprendizado, invenção e/ou significação

“…Os procedimentos metodológicos adotados buscam: analisar e resumir as principais ideias relacionadas a "experiência na educação", contidas na obra de John Dewey (2010;2011a;2011b); analisar e resumir as principais ideias relacionadas ao "processo de individuação", contidas na obra de Gilbert Simondon (2007;2009;2012;; propor aproximações e contraposições entre a categoria "meio comum" (homem, processo cognitivo, máquina e situação), a categoria "código e informação" (ruído, regularidade e acaso), a categoria "experiência e/ou individuação" (interação, continuidade, confronto e eleição) e a categoria "aprendizado, invenção e/ou significação" (percepção, análise, síntese, sincronia e revelação); propor parâmetros capazes de contribuir com a concepção e a elaboração de um "modelo de visualização de dados" sobre a história do design (Carpendale, 2013;Carusi, 2010;Drucker, 2011;Zuk, 2006a;Zuk, 2006b); e finalmente, formalizar a análise e a descrição dos resultados teóricos obtidos na pesquisa.…”