Visualizing a Moving Target: A Design Study on Task Parallel Programs in the Presence of Evolving Data and Concerns

Abstract

Fig. 1. Design Study timeline (log scale). The top contains a mark for each collected artifact. Connections to identified goals, sub-goals, and tasks are marked when direct evidence for them has been identified. Artifacts from meetings presenting major design changes and notes from the evaluation sessions of Section 7.2 are indicated with color. The bottom shows the timing of various deployments with users. This rich collection of over 150 artifacts mitigated issues in designing around shifting data and concerns.Abstract-Common pitfalls in visualization projects include lack of data availability and the domain users' needs and focus changing too rapidly for the design process to complete. While it is often prudent to avoid such projects, we argue it can be beneficial to engage them in some cases as the visualization process can help refine data collection, solving a "chicken and egg" problem of having the data and tools to analyze it. We found this to be the case in the domain of task parallel computing where such data and tooling is an open area of research. Despite these hurdles, we conducted a design study. Through a tightly-coupled iterative design process, we built Atria, a multi-view execution graph visualization to support performance analysis. Atria simplifies the initial representation of the execution graph by aggregating nodes as related to their line of code. We deployed Atria on multiple platforms, some requiring design alteration. We describe how we adapted the design study methodology to the "moving target" of both the data and the domain experts' concerns and how this movement kept both the visualization and programming project healthy. We reflect on our process and discuss what factors allow the project to be successful in the presence of changing data and user needs.

Guidelines For Pursuing and Revealing Data Abstractions

“…Our data (C22) validates the known pitfall [38] in which the lack of access to real data can doom a design study collaboration, because visualization researchers are less likely to have enough actionable information to articulate an accurate data abstraction. It also validates that a culture of data review [46], that is careful to emphasize good communication and transparency about the data abstraction, can compensate for a lack of access to real data because the detailed abstraction is a joint objective that all parties have a stake in.…”

CcNav: Understanding Compiler Optimizations in Binary Code

“…Prototyping the visualization was done in tandem with prototyping the automated analysis. This is also a prototyping activity as noted by Hall et al and adds a "moving target" element as discussed by Williams et al [47]. As noted by Williams et al, the copious documentation of tasks and interests over time helped us to prioritize design elements that fulfilled long-standing task-needs over those that had gained attention fleetingly.…”

“…Focus on particular optimizations or analysis strategies varied from meeting to meeting, though the overall goal did not change. Similar to Williams et al [47], we used the persistence of tasks over time to prioritize the design and implementation of CcNav features. Fig.…”

Data-First Visualization Design Studies

“…Launching a design study project without access to real data is a common pitfall [36]. Synthetic or toy data often results in additional project obstacles and delays [18,39], and may even lead down the wrong path of data abstraction and visualization design. Access to real-world datasets at the beginning of the project allows the creation of data sketches [21] and technology probes [16], both to internally guide the development of hypotheses and possible usage scenarios, and to externally inquire needs and desires of target user groups.…”