A Multiverse Analysis on the relationship between interdisciplinary research and scientific impact

Abstract

Interdisciplinary research is a key issue in science policy because it is commonly associated with scientific advances and societal impact. However, recent research has shown substantial problems with underlying metrics, measures and models. As with many areas of innovation studies, and the wider social sciences, the many ways it can be defined and measured create excessive ‘researcher degrees of freedom’ raising questions about the robustness of findings. Robustness checks reduce but do not eliminate this problem. This article builds on this work and conducts a multiverse analysis of 19,028 articles from 89 prestigious business journals to assess and quantify the sensitivity of regression coefficients to factors of variations in 6,480 marginally differing models of the relationship between interdisciplinarity and future citations. The results indicate a weak positive effect on scientific impact, albeit with a noticeable time lag. The article introduces a new measure of disciplinary divergence which has a more pronounced and robust influence on scientific impact than previous diversity-based measures, and is more robust to changes in taxonomic granularity. The article formally defines the mathematical differences among indicators of semantic atypicality through a novel method for calibrating the Disparity of semantic measurements. This approach helps to explain the paradoxical findings of previous studies that disciplinary Variety is positively related to scientific impact, but disciplinary Balance is not.