Robustness, Diversity of Evidence, and Probabilistic Independence

Abstract

In robustness analysis, hypotheses are supported to the extent that a result proves robust, and a result is robust to the extent that we detect it in diverse ways. But what precise sense of diversity is at work here? In this paper, I show that the formal explications of evidential diversity most often appealed to in work on robustnesswhich all draw in one way or another on probabilistic independence -fail to shed light on the notion of diversity relevant to robustness analysis. I close by briefly outlining a promising alternative approach inspired by Horwich's (1982) eliminative account of evidential diversity. Robustness Analysis in ScienceTo verify that results are not simply artifacts of the particular means used to detect them, scientists often attempt to duplicate those results using other, diverse means. To the extent that a result is detected via numerous, diverse means, it is said to be robust. Robustness analysis (henceforth, "RA") is a mode of reasoning in which one supports a hypothesis via an analysis of the conditions under which a result is robust.Examples of RA from scientific practice abound. Famously, biologist Richard Levins (1966) proposes RA as a general means for deciphering, when using simplified models to study complex systems, whether a result "depends on the essentials of a model or on the details of the simplifying assumptions:"[W]e attempt to treat the same problem with several alternative models each with different simplifications but with a common biological assumption. Then, if these models, despite their different assumptions, lead to similar results we have what we can call a robust theorem which is relatively free of the details of the model. Hence our truth is the intersection of independent lies.

How field office leaders drive learning and creativity in humanitarian aid: Exploring the role of boundary‐spanning leadership for expatriate and local aid worker collaboration

“…Compared with other humanitarian studies featuring samples—such as 141 former Nepalese child soldiers versus nonsoldiers (Kohrt et al, ), 132 aid workers across 22 countries (Moshtari, ), or 152 NGO leaders (Mitchell, )—our sample is comparable in size and sufficiently diverse. As such, it provides a rather robust test of our hypotheses (see Schupbach, ).…”

“…In fact, according to Iacobucci's () methodological guidelines for behavioral researchers, our structural equation modeling‐based moderated mediation can perform well with the current sample size. Moreover, because data encompassed 59 field offices across various countries and operations, the present sample was sufficiently diverse to support a rather robust test of the hypotheses (see Schupbach, ). Nonetheless, an increase in the sample size could enable potentially valuable subsample analyses that the current sample size did not reliably permit.…”

Vindicating methodological triangulation

“…To put the point more positively, if we suspect that two approaches we have so far counted as different methods (almost) always give the same answer, then we should count these as only one method for the purpose of triangulation. In practice it may be difficult to identify whether methods are independent in the relevant sense (Stegenga 2012;Schupbach 2015). But Kuorikoski and Marchionni (2016) provide evidence that such independence holds in at least some cases.…”

The variety-of-evidence thesis: a Bayesian exploration of its surprising failures

“…3 For recent attempts to spell out a complete characterization of evidential variety in arguments relying on evidence amalgamation, see Schupbach (2015); Kuorikoski and Marchionni (2016). Assessing these propositions falls outside the scope of our article.…”

“…1 The most recent research includes Schupbach (2015); Kuorikoski and Marchionni (2016); Stegenga and Menon (2017).…”