Research and development in computer science and statistics have produced increasingly sophisticated software interfaces for interactive and exploratory analysis, optimized for easy pattern finding and data exposure. But design philosophies that emphasize exploration over other phases of analysis risk confusing a need for flexibility with a conclusion that exploratory visual analysis is inherently "model free" and cannot be formalized. We describe how without a grounding in theories of human statistical inference, research in exploratory visual analysis can lead to contradictory interface objectives and representations of uncertainty that can discourage users from drawing valid inferences. We discuss how the concept of a model check in a Bayesian statistical framework unites exploratory and confirmatory analysis, and how this understanding relates to other proposed theories of graphical inference. Viewing interactive analysis as driven by model checks suggests new directions for software and empirical research around exploratory and visual analysis. For example, systems should enable specifying and explicitly comparing data to null and other reference distributions and better representations of uncertainty. Implications of Bayesian and other theories of graphical inference should be tested against outcomes of interactive analysis by people to drive theory development.Media Summary: Novel interactive graphical user interface tools for exploratory visual data analysis provide analysts with impressive flexibility in how to look at and interact with data. Often these systems are designed to make patterns in data as easy to see as possible. However, there are risks to designing systems for easy pattern finding alone. One risk is that the techniques used to emphasize patterns, like aggregating data by default, lead analysts to overlook variation and uncertainty in their data, leading them to draw conclusions that aren't well supported by the data. Another is that some analysts may fail to recognize the importance of confirming any insights they arrive at through visual search using confirmatory statistical modeling to make sure they are valid. One reason that graphical user interface systems for interactive analysis may not be designed to enforce strong connections between exploratory and confirmatory analysis is because there aren't well-established theories of how these two types of activities are related. We propose a perspective that unites exploratory and confirmatory analysis through the idea of graphs as model checks in a Bayesian statistical framework, and describe how in light of this view, it becomes clear that systems for exploratory visual analysis should better support model-driven inference and representation of uncertainty.