Quantification of Unknown Unknowns in Astronomy and Physics

Abstract

Uncertainty quantification is a key part of astronomy and physics; scientific researchers attempt to model both statistical and systematic uncertainties in their data as best as possible, often using a Bayesian framework. Decisions might then be made on the resulting uncertainty quantificationperhaps whether or not to believe in a certain theory, or whether to take certain actions. However it is well known that most statistical claims should be taken contextually; even if certain models are excluded at a very high degree of confidence, researchers are typically aware there may be systematics that were not accounted for, and thus typically will require confirmation from multiple independent sources before any novel results are truly accepted. In this paper we compare two methods in the astronomical literature that seek to attempt to quantify these 'unknown unknowns' -in particular attempting to produce realistic thick tails in the posterior of parameter estimation problems, that account for the possible existence of very large unknown effects. We test these methods on a series of case studies, and discuss how robust these methods would be in the presence of malicious interference with the scientific data.

Promotion and caution in research article abstracts: The use of positive, negative and hedge words across disciplines and rankings

“…This disciplinary propensity has to do with different paradigms of the social construction of knowledge that hedges are used more frequently in soft sciences due to the fact that 'soft-knowledge areas however are typically more interpretative and less abstract' characterized by heterogeneity at various contextual levels such as 'less control of variables', 'more diversity of research outcomes', and negotiations of arguments between writers and readers (Hyland, 1998b, p. 361). This argument could also explain our results that more hedge words were used in astronomy, a hard science field, because apart from large amount of 'unknown unknowns', researchers in astronomy may mainly deal with 'known unknowns', the sources of uncertainty in the analysis including statistical, systematic, numerical, and so on (Hatfield, 2022). These uncertainties might result in more use of hedge words in astronomy.…”