Securing the Empirical Value of Measurement Results

Abstract

Reports of quantitative experimental results often distinguish between the statistical uncertainty and the systematic uncertainty that characterize measurement outcomes. This paper discusses the practice of estimating systematic uncertainty in High Energy Physics (HEP). The estimation of systematic uncertainty in HEP should be understood as a minimal form of quantitative robustness analysis. The secure evidence framework is used to explain the epistemic significance of robustness analysis. However, the empirical value of a measurement result depends crucially not only on the resulting systematic uncertainty estimate, but on the learning aims for which that result will be used. Philosophically important conceptual and practical questions regarding systematic uncertainty assessment call for further investigation.

Why Trust a Simulation? Models, Parameters, and Robustness in Simulation-Infected Experiments

“…Like Staley (2020), I will use experimental High energy physics (HEP), the study of interactions between material particles in high-energetic collisions, as a case study; for Morrison (2015, p. 287) displays CSs as a conditio sine qua non therein: They are used to estimate the discovery potential of detector components before these are built, define the 'background' data to be expected in the absence of phenomena of interest for statistical analyses, or even what signatures of interest might look like. Thus, "simulation plays a significant role in the experimental setting" (Morrison, 2015, p. 288, orig.…”

“…RA has been applied to models (e.g. Weisberg, 2006;Weisberg and Reisman, 2008;Lloyd, 2009Lloyd, , 2010Parker, 2011), experiments (Staley, 2004b(Staley, , 2020Schupbach, 2016;Karaca, 2018), and CSs (Muldoon, 2007;Durán and Formanek, 2018;Gueguen, 2019;Lehtinen and Kuorikoski, 2007). In HEP, these issues become entangled, and subjecting models and simulation to RA can in principle serve the goal of inferring something about experimental results or the status of simulation models.…”

“…In a recent paper, Staley (2020) has argued that such results gain their credibility in part from being associated with a measure of systematic uncertainty, which can be fruitfully viewed as the result of a RA, reflecting the impact of choices in the modelling-assumptions facilitating the inference to the desired result. 4 This is consistent with Karaca's (2018, p. 7) verdict that robustness "comes in degrees in the sense that its extent indicates the extent of convergence of [...] results obtained through different means of detection."…”

“…Staley presupposes what he calls the 'secure evidence framework' (Staley, 2014(Staley, , 2020, according to which an epistemic agent needs to evaluate claim, C, that data x count as evidence for hypothesis H in relation to her epistemic situation K and an induced set of epistemically possible scenarios Ω. Defining the result's systematic and statistical uncertainty is key to weakening C, which, equivalently, means rendering it independent from potentially erroneous assumptions in Ω.…”

“…1-2). The resulting probability-weights (called 'parton density functions'), furthermore, can only be obtained by fitting parametric functions to a broad range of data (also Staley, 2020;Ritson and Staley, 2020). Similarly, shower models utilize an approximate QCD-result as the basis of a Markovchain in which partons are radiated at distinct times (cf.…”