Here’s what you need to know about my data: Exploring Expert Knowledge’s Role in Data Analysis.

Abstract

Data driven decision making has become the gold standard in science, industry, and public policy. Yet data alone, as an imperfect and partial representation of reality, is often insufficient to make good analysis decisions. Knowledge about the context of a dataset, its strengths and weaknesses, and its applicability for certain tasks is essential. In this work, we present an interview study with analysts from a wide range of domains and with varied expertise and experience inquiring about the role of contextual knowledge. We provide insights into how data is insufficient in analysts workflows and how they incorporate other sources of knowledge into their analysis. We also suggest design opportunities to better and more robustly consider both, knowledge and data in analysis processes.

“Two Heads are Better than One”: Pair-Interviews for Visualization

“…We draw our recommendations from four separate pair-interview teams, all of which conducted semi-structured interviews in an academic setting with graduate students or postdocs as the interviewers. The interviews were conducted across a variety of topics and participants: families with asthmatic kids interacting with data from air quality sensors in their homes [15] (S1); visualization design study researchers reflecting on the end of their collaborations [1] (S2); employees at a manufacturing company transitioning to a commercial dashboarding system [21] (S3); and data workers in a broad range of fields discussing how they deal with imperfect data [13] (S4). We provide a brief sketch of the different studies in Table 1 to illustrate the range of visualization contexts these pair-interviews took place within.…”

“…The Interviewers column illustrates the composition of the interview team. To read more about the studies: S1 [15]; S2 [1]; S3 [21]; S4 [13].…”

“…Pair-interviews are interviews conducted by two people, rather than one, with the responsibilities of planning, conducting, and analyzing the interviews strategically divided between two complementary roles: a driver and a navigator. By reflecting across the use of pair-interviews within four separate visualization studies [1,13,15, 21] -Akbaba was involved in two of the studies, and Meyer in all four -we find that the explicit enactment of these roles reduced the logistical and emotional burdens of conducting an interview, increased the engagement with the interviewee, and improved the experience of interview analysis through co-reflection. As a method that focuses on the mechanics of asking questions and directing discussions during an interview, pair-interviewing complements other interview methods used by visualization researchers that instead focus on the interview questions [9], the interview context [5,10], data analysis during the interview [3,16], or participant interactions with technologies [19].…”