Design and Comparative Evaluation of Visualization Onboarding Methods

Abstract

Comprehending and exploring large and complex data is becoming increasingly important for a diverse population of users in a wide range of application domains. Visualization has proven to be well-suited in supporting this endeavor by tapping into the power of human visual perception. However, nonexperts in the field of visual data analysis often have problems with correctly reading and interpreting information from visualization idioms that are new to them. To support novices in learning how to use new digital technologies, the concept of onboarding has been successfully applied in other fields and first approaches also exist in the visualization domain. However, empirical evidence on the e ectiveness of such approaches is scarce. Therefore, we conducted three studies with Amazon Mechanical Turk (MTurk) workers and students investigating visualization onboarding at di erent levels: (1) Firstly, we explored the e ect of visualization onboarding, using an interactive step-by-step guide, on user performance for four increasingly complex visualization techniques with time-oriented data: a bar chart, a horizon graph, a change matrix, and a parallel coordinates plot. We performed a between-subject experiment with 596 participants in total. The results showed that there are no significant di erences between the answer correctness of the questions with and without onboarding. Furthermore, participants commented that for highly familiar visualization types no onboarding is needed. ( 2) We performed a second study with MTurk workers to assess if there is a di erence in user performances on di erent visualization onboarding types: step-by-step, scrollytelling tutorial, and video tutorial. The study revealed that the video tutorial was ranked as the most positive on average, based on a sentiment analysis, followed by the scrollytelling tutorial and the interactive step-by-step guide. (3) For our third study, we gathered data towards users' experience in using the in-situ scrollytelling for the VA tool Netflower. The results of the evaluation with students showed that they preferred scrollytelling over the tutorial integrated in the Netflower landing page. Moreover, for all three studies we explored the e ect of task di culty. In summary, the in-situ scrollytelling approach works well for integrating onboarding in a visualization tool. Additionally, a video tutorial can help to introduce interaction techniques of a visualization.

Transitioning to a Commercial Dashboarding System: Socio-Technical Observations and Opportunities

“…Also referred to as narrative visualization, this work investigates how to combine visualizations with annotations, narrations, and interaction techniques to communicate ideas from and with data. Building from the initial characterization of data-driven storytelling by Segel & Heer [49], more recent work in the space has explored combining comics and data [7,8], accessibility through automatic audio narrations [51], scrollytelling with interactive visualizations [39,53], and techniques for virtual narration of charts [15,30]. This line of research has further extended into data videos that augment visualizations with animations, audio and text narrations, music, and sound effects to elicit engagements with data [4,5,45,50,59].…”

A Process Model for Dashboard Onboarding

“…Regardless of the exact means chosen, visualization onboarding is often based on a narrative. Stoiber et al [SWG * 21], for example, separated their onboarding instructions for single visualizations into reading the chart, interacting with the chart, and using the chart. In the course of conceptual work for our process model, we realized that an onboarding narrative is particularly important for dashboards, as they include multiple visualizations and potentially complex interactions.…”

“…Based on the literature [CFGT21], our own experience from previous projects related to onboarding [SGP * 19, SWG * 21], and discussions with collaboration partners from the steel and pharmaceutical industries who create and use dashboards, we know that the onboarding process can vary based on the target groups (characterized by differences in dashboard literacy). Some users may require in‐depth onboarding to a specific visualization type with an explanation of the data sources and the model, while others require only a high‐level understanding of the visualizations and how they are linked.…”

Educational Data Comics: What can Comics do for Education in Visualization?

“…This can be a problem considering that a lot of data can only be thoroughly explored through less widely known, possibly even interactive visualization techniques, even though the interpretation of such data is vital to decision-making. While there is a growing literature on education methods, including workshops [53], evaluation methods [11], onboarding [45,46], visualization literacy assessment tests [35], and teaching approaches [9,24], we need to focus on providing efficient and effective resources to learn visualizations outside traditional classroom settings and in situations of informal learning that occur away from a structured, formal classroom environment: self-learning, online learning, and professional development.…”