Click2Annotate: Automated Insight Externalization with rich semantics

Abstract

Insight Externalization (IE) refers to the process of capturing and recording the semantics of insights in decision making and problem solving. To reduce human effort, Automated Insight Externalization (AIE) is desired. Most existing IE approaches achieve automation by capturing events (e.g., clicks and key presses) or actions (e.g., panning and zooming). In this paper, we propose a novel AIE approach named Click2Annotate. It allows semi-automatic insight annotation that captures low-level analytics task results (e.g., clusters and outliers), which have higher semantic richness and abstraction levels than actions and events. Click2Annotate has two significant benefits. First, it reduces human effort required in IE and generates annotations easy to understand. Second, the rich semantic information encoded in the annotations enables various insight management activities, such as insight browsing and insight retrieval. We present a formal user study that proved this first benefit. We also illustrate the second benefit by presenting the novel insight management activities we developed based on Click2Annotate, namely scented insight browsing and faceted insight search.

An Affordance-Based Framework for Human Computation and Human-Computer Collaboration

“…In presenting a preliminary framework for describing and comparing systems involving human and machine collaborators, we aspire to lay the foundation for a more rigorous analysis of the tools and approaches presented by our field, thereby paving the way for the construction of an increasingly robust understanding of analytical reasoning and how to best support insight generation. [41] CrowdSearch [80] ParallelTopics [23] Dissimilarity [50] VH+ML [28] Implicit tagging [62] reCAPTCHA [77] VizWiz [10] Phetch [74] ESP Game [73] KissKissBan [33] LabelMe [59] Ka-captcha [21] PeekABoom [76] MRI [12] iView [83] iVisClassifier [18] Saliency [37] RP Explorer [3] DimStiller [36] WireVis [46] Action trails [65] NetClinic [47] Trajectories [4] Risk assessment [51] Automatic transfer functions [57] MDX [66] Automated+viz [68] CzSaw [39] Fold.it [20] HRI scripts [17] Animated agents for VR [56] VA Model-learning [29] EyeSpy [6] MonoTrans2 [35] CastingWords [16] Click2Annotate [15] Wrangler [40] Soylent [8] Crowdsourced solutions [67] Crowdsourced design [81] Stress OutSourced …”

“…It has also been utilized to refine classification models and performing dimension reduction [18,29,51], interactively cluster data [4], and automatically extract transfer functions from user-selected data [57]. It has been used to suggest informative data views [83], and even to help users externalize and understand their own insight generation process [15,39,41,46,65].…”

“…The study of general human-computer collaboration offers a plethora of examples of successful human/machine teams [15,23,37,39,41,46,50,65,66,68,83]. Developments in supervised machine learning in the visualization community present several vetted techniques for human intervention into computationally complex tasks [3,4,12,18,28,29,36,47,51,57].…”

Supporting effective common ground construction in Asynchronous Collaborative Visual Analytics

“…We have implemented our common ground construction approaches in ManyInsights, a web-based system for asynchronous collaborative analysis of multidimensional data [6]. Our previous work on semi-automatic insight annotation, named Click2Annotate [6], is integrated in ManyInsights and used as a basis of the proposed approaches in this paper.…”

“…), and meta information (e.g., authors, timestamps, etc. ), from semi-automatically generated, formalized insight annotations [6]. A rich set of views and interactions built upon automatic insight analysis are then provided.…”