Effect of Adaptive Guidance and Visualization Literacy on Gaze Attentive Behaviors and Sequential Patterns on Magazine-Style Narrative Visualizations

Abstract

We study the effectiveness of adaptive interventions at helping users process textual documents with embedded visualizations, a form of multimodal documents known as Magazine-Style Narrative Visualizations (MSNVs). The interventions are meant to dynamically highlight in the visualization the datapoints that are described in the textual sentence currently being read by the user, as captured by eye-tracking. These interventions were previously evaluated in two user studies that involved 98 participants reading excerpts of real-world MSNVs during a 1-hour session. Participants’ outcomes included their subjective feedback about the guidance, and well as their reading time and score on a set of comprehension questions. Results showed that the interventions can increase comprehension of the MSNV excerpts for users with lower levels of a cognitive skill known as visualization literacy. In this article, we aim to further investigate this result by leveraging eye-tracking to analyze in depth how the participants processed the interventions depending on their levels of visualization literacy. We first analyzed summative gaze metrics that capture how users process and integrate the key components of the narrative visualizations. Second, we mined the salient patterns in the users’ scanpaths to contextualize how users sequentially process these components. Results indicate that the interventions succeed in guiding attention to salient components of the narrative visualizations, especially by generating more transitions between key components of the visualization (i.e., datapoints, labels, and legend), as well as between the two modalities (text and visualization). We also show that the interventions help users with lower levels of visualization literacy to better map datapoints to the legend, which likely contributed to their improved comprehension of the documents. These findings shed light on how adaptive interventions help users with different levels of visualization literacy, informing the design of personalized narrative visualizations.

Interactive visualization literacy: The state-of-the-art

“…They are requested to analyze their visual designs verbally. The study ends with an interview questionnaire to explore the resulting 26 35-50, 39 -Huron et al 37 22-43, 28 70 min Lalle ´et al 49 18-69, 26 90 min Huynh et al 53 11-13, -60 min Barral et al 56 19-69, 26, 115 min Barral et al 57 19-69, 26, 115 min visual layouts and problems encountered while students were constructing visual designs. This study reports three visual mapping process barriers: (i) selecting the parameters to map to the visual variables, (ii) selecting the visual marks to be used, and (iii) decoding and interpreting the visual result.…”

“…Wang et al 51 Olympics, Iris, Cliques, Clusters, or Bridges Rodrigues et al 52 Mock Data Huynh et al 53 Constellations, Fictional Data Fictional Characters D'Ignazio and Bhargava 54 Data-driven Story Donohoe and Costello 55 Oil Price, Internet speed, Cost of Food, etc. Barral et al 56 Car Sales, Syrian Refugees, World Economy Barral et al 57 Car Sales, Syrian Refugees, World Economy Peppler et al 58 Personal data 35 ü High S., Higher E. Alper et al 41 ü Primary School Fuchs et al 47 ü Higher Education Ga ¨bler et al 48 ü High School Bishop et al 27 ü Primary School Krekhov et al 50 ü Higher Education Firat et al 22 ü Higher Education Wang et al 51 ü Higher Education Rodrigues et al 52 ü Higher Education Total: 10 8 2…”

“…Baker et al 34 R + W A 52 Delmas et al 35 R A 1464 Scho ¨nborn et al 36 R A + C N/A Grammel et al 25 R + W A 4, 9 Kodagoda et al 26 R A 24 Boy et al 4 R A 34, 37, 34, 39 Huron et al 37 R + W A 12 Ruchikarhorn and Mueller 38 R A 22, 11,11 Maltese et al 39 R A 202 Kwon and Lee 40 R + W A 120 Bo ¨rner et al 5 R A + C 273 Alper et al 41 R + W C 6, 15 Lee et al 6 R A 65, 191 Wojton et al 42 R A 388 Chevalier et al 1 R + W C N/A Zoss et al 43 R + W A+ C N/A Mansoor and Harrison 44 R A + C N/A Bo ¨rner et al 45 R + W A+ C N/A Stoiber et al 46 R A + C N/A Lee et al 28 R A 178 Fuchs et al 47 R A 28 Ga ¨bler et al 48 R C 23 Bishop et al 27 R C 24 Lalle ´et al 49 R A 119 Krekhov et al 50 R + W A 11 Firat et al 22 R A 25 Wang et al 51 R + W A 11 Rodrigues et al 52 R A 22 Huynh et al 53 R C 33 D'Ignazio and Bhargava 54 R + W A + C N/A Donohoe and Costello 55 R A 32 Barral et al 56 R A 56, 119 Barral et al 57 R A 56, 119 Peppler et al 58 R The first in the wild concentrates on the process of collaborating with community groups, Groundwork Somerville, and local youth design to paint a datadriven story as a community mural. This example of a ''data mural'' documents an action-oriented, community-based project that builds data literacy.…”

“…Barral et al 56 expand this research by identifying the particular visual behaviors that adaptive guidance produces in users, based on their level of visualization literacy. Barral et al 57 extend their previous user study 49 by examining the speed with which users process newly triggered intervention in order to understand how effective interventions are to direct the attention of the user in the visualization and how this adaptive mechanism influences the users according to their visual literacy levels.…”

Scoping the Future of Visualization Literacy: A Review

“…it did not discuss the believability or construction of visualizations. Effect of Adaptive Guidance and Visualization Literacy on Gaze Attentive Behaviors and Sequential Patterns on Magazine-Style Narrative Visualizations [2] used visualization literacy as an independent variable to find ways to support those with low visualization literacy skills, but did not develop the research of visualization literacy itself. The Correlation between Users' Cognitive Characteristics and Visualization Literacy [22] discussed visualization literacy at an individual level using cognitive characteristics which was not the focus of this study.…”