SketchInsight: Natural data exploration on interactive whiteboards leveraging pen and touch interaction

Abstract

In this work, we advance research efforts in combining the casual sketching approach of whiteboards with the machine's computing power. We present SketchInsight, a system that applies the familiar and collaborative features of a whiteboard interface to the accurate data exploration capabilities of interactive visualizations. SketchInsight enables data analysis with more fluid interaction, allowing people to visually explore their data by drawing simple charts and directly manipulating them. In addition, we report results from a qualitative study conducted to evaluate user experience in exploring data with SketchInsight, expanding our understanding on how people use a pen-and touch-enabled digital whiteboard for data exploration. We also discuss the challenges in building a working system that supports data analytic capabilities with pen and touch interaction and freeform annotation.

Post-WIMP Interaction for Information Visualization

“…SketchStory (Lee et al, 2013) demonstrated how freeform sketching with a pen coupled with simple touch interactions can be leveraged to create engaging data-driven presentations. PanoramicData (Zgraggen et al, 2014) and SketchInsight (Lee et al, 2015) (Figure 2.3b), conversely, showed how a combination of pen and touch can enable more natural data exploration on an infinite canvas. We note that prior research was conducted to inform the design of SketchInsight before more capable devices became available.…”

Reaching Broader Audiences With Data Visualization

“…To investigate the potential benefits of leveraging multiple input modalities, several visualization systems have combined at least two modes of input. For example, SketchInsight allows people to visually explore their data by interactively creating and manipulating charts with pen and touch interaction [16]. More recently, Orko (Fig-ure 4) combines speech-based natural language with touch-based direct manipulation to facilitate network-based data exploration [15].…”

ActiveInk

“…There have been several approaches to design a coherent set of touch gestures for interacting with visualizations on tablets [8,11,[25][26][27], but these do not fully take advantage of the different inking capabilities. Visualization systems such as SketchInsight [15], PanoramicData [40], and Viz-Dom [7] offer novel experiences leveraging pen and touch interactions for data analysis tasks. The research focus of these systems is on the design of interactions to create and interact with visualizations.…”

“…These devices offer much potential for knowledge workers, affording large workspaces to handle multiple charts and documents, and natural interactions to manipulate and annotate them. Visualization researchers have started to explore this technology in systems tailored for pen and touch interactions (e.g., SketchInsight [15], Viz-Dom [7], DataInk [38]). But these previous efforts have focused much less on how to enable people to externalize their thoughts as they engage in sensemaking activities.…”