Vindicating methodological triangulation

Abstract

Social scientists use many different methods, and there are often substantial disagreements about which method is appropriate for a given research question. In response to this uncertainty about the relative merits of different methods, W. E. B. Du Bois advocated for and applied "methodological triangulation". This is to use multiple methods simultaneously in the belief that, where one is uncertain about the reliability of any given method, if multiple methods yield the same answer that answer is confirmed more strongly than it could have been by any single method. Against this, methodological purists believe that one should choose a single appropriate method and stick with it. Using tools from voting theory, we show Du Boisian methodological triangulation to be more likely to yield the correct answer than purism, assuming the Thanks to Natalie Ashton, Seamus Bradley, Clark Glymour, Chike Jeffers, Aidan Kestigian, Erich Kummerfeld, Christian List, Wendy Parker, Kevin Zollman, two anonymous reviewers, and audiences in Munich and London for helpful comments. RH and LKB acknowledge support from the National Science Foundation through grant SES 1254291. RH also acknowledges support from the Leverhulme Trust and the Isaac Newton Trust through an Early Career Fellowship. 123Synthese scientist is subject to some degree of diffidence about the relative merits of the various methods. This holds even when in fact only one of the methods is appropriate for the given research question.

In Epistemic Networks, Is Less Really More?

“…Some maintain that robust results in the sense just described are better supported (Levins, 1966;Wimsatt, 2012;Weisberg, 2006;Heesen et al, 2014). Others hold that this sort of robustness analysis, in fact, cannot support modeling results (Orzack and Sober, 1993;Odenbaugh and Alexandrova, 2011;Cartwright, 1991).…”

Evidence amalgamation in the sciences: an introduction

“…Heesen et al (2018) employ a triangulation strategy while Bovens and Hartmann (2003) update using Bayesian conditionalization. Those with very strong intuitions for the VET may thus feel compelled to give up on Bayesian updating.…”

“…Furthermore, they consider consequence variables and reliability variables that are dependent to a degree, showing that the VET fails in many of their models. Heesen et al (2018) break with the tradition of a Bayesian analysis of the confirmatory value of varied evidence obtained by employing diverse methods. In their contribution drawing on voting theory, it is the evidence from diverse methods which supports the hypothesis of interest more strongly.…”

“…Consequently, the conclusions drawn for the thesis are less favorable in Claveau and Grenier (2018) than in Claveau (2013). Heesen et al (2018) are interested in the work of Du Bois (1996/1899) concerning the question, "What do Negroes earn?" (To avoid anachronism, they follow Du Bois' terminology in using "Negro" rather than the more contemporary "African-American.")…”

“…They are then interested in how to aggregate answers obtained from these different methods. Heesen et al (2018) delineate (i) a purist strategy-single out a method and believe whatever this method finds-from (ii) a triangulation strategy-believe what most methods find and break ties by randomly picking from best supported findings. They show that the triangulation strategy has a greater probability of yielding the correct answer than the purist strategy for m ≥ 3 methods and n ≥ 2 possible answers.…”