A Grammar for Hypothesis-Driven Visual Analysis

Abstract

A hallmark of visual analytics is its ability to support users in translating broad, open-ended analytic questions (e.g., "is our company succeeding?") into verifiable hypotheses that can be tested from the available data (e.g., "our total revenue increased this past quarter"). However, the process of converting open-ended analytic questions into testable hypotheses is complex and often ill-defined. Beyond high-level descriptions, the visual analytics literature lacks a formalization that can be operationalized for analysis and automation. In this paper, we propose a novel grammar to express hypothesis-based analytic questions for visual analysis. Drawing from prior work in science and education, our grammar defines a formal way to express sets of verifiable hypotheses as a "hypothesis space". Our proposed use of hypothesis spaces contributes a new lens to unify concepts of user goals, the capabilities of a dataset, visual analysis, and testable hypotheses. As a result, we can reformulate abstract classes of visual analysis goals, such as analytic and data-related tasks, in a way that is suitable for analysis and automation. We demonstrate use cases of our grammar in real-world analytic applications including VAST challenges, Kaggle competitions, and pre-existing task taxonomies. Finally, we provide design opportunities in which our grammar can be operationalized to articulate analysis tasks, evaluate visualization systems, and support hypothesis-based reasoning in visual analytic tools.

Visual Analytics for Robust Investigations of Placental Aquaporin Gene Expression in Response to Maternal SARS-CoV-2 Infection

“…The visual analytics approach helped connect placental gene expression to placental biological processes involving aquaporins and other gene groups. Future studies with the datasets could consider hypothesis-driven analysis of the datasets and grammar-based approaches to a hypothesis-driven theory of visual analytics [94,95]. We generated volcano plots, box plots, enclosure data tables, and heatmaps to visualize the RNA-Seq gene expression data.…”

Analysis Without Data: Teaching Students to Tackle the VAST Challenge

“…This process was repeated until participants reported they had done all they could with hypothesis generation without performing data exploration or analysis. Turning Open-ended Questions into Formal Hypotheses: To ensure that the participants were forming meaningful analytic questions and hypotheses that could eventually be tested to solve the challenge, the participants were asked to refine any open-ended questions into formalized hypotheses using either the hypothesis grammar by Suh et al [26] or the Hypothesis Quality Scale by Sunal and Haas [27] (see Table 1). An example of this process is discussed in Section 4.3.…”

“…This approach of methodologically performing data analysis is inspired by recent work in pre-registering hypotheses before visual analysis [12,26] and experimental designs [4]. In experimental studies, pre-registration can prevent p-hacking [7], HARKing (Hypothesizing After Results are Known) [11], and postdiction (as opposed to prediction) [18].…”

Q4EDA: A Novel Strategy for Textual Information Retrieval Based on User Interactions with Visual Representations of Time Series

“…Furthermore, these tools use the query to retrieve data from domain-specific databases, which differs from our proposed query conversion method for search engines. Moreover, although research has identified other ways to encode visual findings and hypotheses beyond visual query constructors [22], no existing work, as far as the authors know, has demonstrated ways to use such methodology to convert a visual selection into a search query as Q4EDA proposes.…”