A Dialogue with the Data: The Bayesian Foundations of Iterative Research in Qualitative Social Science

Abstract

We advance efforts to explicate and improve inference in qualitative research that iterates between theory development, data collection, and data analysis, rather than proceeding linearly from hypothesizing to testing. We draw on the school of Bayesian “probability as extended logic,” where probabilities represent rational degrees of belief in propositions given limited information, to provide a solid foundation for iterative research that has been lacking in the qualitative methods literature. We argue that mechanisms for distinguishing exploratory from confirmatory stages of analysis that have been suggested in the context of APSA’s DA-RT transparency initiative are unnecessary for qualitative research that is guided by logical Bayesianism, because new evidence has no special status relative to old evidence for testing hypotheses within this inferential framework. Bayesian probability not only fits naturally with how we intuitively move back and forth between theory and data, but also provides a framework for rational reasoning that mitigates confirmation bias and ad-hoc hypothesizing—two common problems associated with iterative research. Moreover, logical Bayesianism facilitates scrutiny of findings by the academic community for signs of sloppy or motivated reasoning. We illustrate these points with an application to recent research on state building.

‘It's making his bad days into my bad days’: The impact of coronavirus social distancing measures on young carers and young adult carers in the United Kingdom

“…This approach gives a comprehensive understanding of the phenomena and tests the validity of the analysis through the convergence of sources (da Silva Santos et al, 2020). Although the qualitative data gained through the interviews were richer and more expansive, the corresponding data were drawn from the survey and its findings were important in checking and strengthen the understanding within the iterative analytical process (Fairfield & Charman, 2019).…”

Taking back control: comprador bankers and managerial developmentalism in Poland

“…Empirical evidence: bankers' involvement in polish economic policy since the crisis In order to develop my hypothesis, I have used the method of process tracing in an iterative way (Fairfield & Charman, 2019). I describe my methodological approachincluding data sources (interviews, newspapers, etc.)…”

Causal Inference and Policy Evaluation from Case Studies Using Bayesian Process Tracing

“…There is a good rationale for this injunction in frequentist statistical methods, as a theory derived from correlations found in a population sample cannot legitimately be tested against that same population sample, as the probability of disproving the new theory is zero. Using Bayesian logic in PT, however, makes it possible to derive a theory from a piece of evidence and then test that theory in the same case (Fairfield & Charman, 2018). There are two reasons for this, one incontrovertible and one more contestable.…”