“Positive” Results Increase Down the Hierarchy of the Sciences

Abstract

The hypothesis of a Hierarchy of the Sciences with physical sciences at the top, social sciences at the bottom, and biological sciences in-between is nearly 200 years old. This order is intuitive and reflected in many features of academic life, but whether it reflects the “hardness” of scientific research—i.e., the extent to which research questions and results are determined by data and theories as opposed to non-cognitive factors—is controversial. This study analysed 2434 papers published in all disciplines and that declared to have tested a hypothesis. It was determined how many papers reported a “positive” (full or partial) or “negative” support for the tested hypothesis. If the hierarchy hypothesis is correct, then researchers in “softer” sciences should have fewer constraints to their conscious and unconscious biases, and therefore report more positive outcomes. Results confirmed the predictions at all levels considered: discipline, domain and methodology broadly defined. Controlling for observed differences between pure and applied disciplines, and between papers testing one or several hypotheses, the odds of reporting a positive result were around 5 times higher among papers in the disciplines of Psychology and Psychiatry and Economics and Business compared to Space Science, 2.3 times higher in the domain of social sciences compared to the physical sciences, and 3.4 times higher in studies applying behavioural and social methodologies on people compared to physical and chemical studies on non-biological material. In all comparisons, biological studies had intermediate values. These results suggest that the nature of hypotheses tested and the logical and methodological rigour employed to test them vary systematically across disciplines and fields, depending on the complexity of the subject matter and possibly other factors (e.g., a field's level of historical and/or intellectual development). On the other hand, these results support the scientific status of the social sciences against claims that they are completely subjective, by showing that, when they adopt a scientific approach to discovery, they differ from the natural sciences only by a matter of degree.

Can the behavioral sciences self-correct? A social epistemic study

“…This is intuitively expected given the widespread practice of null hypothesis significance testing (NHST). Fanelli (2010) reports shocking metrics of this phenomenon. He analyzed 2434 papers from all disciplines that report hypothesis testing, and shows that as we go down in "the hierarchy of sciences" (i.e., a hypothesized hierarchy that Fanelli traces back to Comte, in which physics is at the top and the behavioral sciences at the bottom-of course, we need not agree that there is such a hierarchy), published results tend to be more significant than non-significant.…”

Meta-assessment of bias in science

“…However, it is yet unknown to what extent bias patterns and postulated risk factors are generalizable phenomena that threaten all scientific fields in similar ways and whether studies documenting such problems are reproducible (5)(6)(7). Indeed, evidence suggests that biases may be heterogeneously distributed in the literature.…”

The ironic effect of significant results on the credibility of multiple-study articles.

“…For example, medical journals have seen an increase in the percentage of retracted articles (Steen, 2011a(Steen, , 2011b, and there is the concern that a vast number of published findings may be false (Ioannidis, 2005). However, a recent comparison of different scientific disciplines suggested that the bias is stronger in psychology than in some of the older and harder scientific disciplines at the top of a hierarchy of sciences (Fanelli, 2010).…”