reVISit: Looking Under the Hood of Interactive Visualization Studies

Abstract

Figure 1: Analysis pipeline for instrumenting empirical user studies with provenance data for flexible analysis of the collected data. During study design (1), the visualizations, tasks, and any controlled variables are defined. The visualization technique is (2) instrumented using provenance and response logging. The (3) user study produces the (4) study data, which is then analyzed in (5) reVISit using a suite of data analysis methods. (6) The outcomes of the analysis process include quality control, detecting participant analysis strategies, and generating new hypotheses. The process can be also be used to refine pilot studies.

Provectories: Embedding-based Analysis of Interaction Provenance Data

“…Analytical Strategies: In line with the observed select and refine analysis strategy identified by Nobre et al [34], we can confirm this strategy for single user sessions by means of the playback function in the history graph. Moreover, when performing single-session investigations for the outlier tasks, a top-to-bottom approach becomes visible (S3 and S4).…”

“…However, they call for future work, as analytic provenance in contrast to data provenance is still in its infancy. With reVISit, Nobre et al [34] analyzed interaction provenance by comparing event sequences using a nodelink diagram and identified "multidrag", "sort and select", and "select and refine" as analysis strategies. Provectories aims to identify such user strategies as visual patterns.…”

“…The Knowledge-Transfer Graph by Zhao et al [54] shows a node-link visualization that aims to help researchers to externalize their thought processes in collaborative analyses. reVISit [34] assesses interaction provenance data based on both qualitative and quantitative data by showing interaction patterns and analysis strategies as event sequences.…”

“…To facilitate tracing of individual user sessions even in the presence of potential visual clutter, the history graph with a list of user actions and the resulting application states for an individual session can be activated (see Figure 5b). In reVISit, Nobre et al [34] refer to the sequential analysis of interaction states by a video-like experience using a playback feature. Selecting states in the history graph overlay the embedding space with arrows that indicate the position and direction of the user session.…”

“…10: On the left, summary visualization for all interaction provenance states within the embedding for the dataset outlier (linear) easy 1. 1 -5 shows the summary visualizations using the storytelling feature (playback features as within Nobre et al [34]) for analyzing a single user session. Outlier were first selected on top before selecting data points at the bottom of the scatterplot and going up again.…”