Willingness to Share Research Data Is Related to the Strength of the Evidence and the Quality of Reporting of Statistical Results

Abstract

BackgroundThe widespread reluctance to share published research data is often hypothesized to be due to the authors' fear that reanalysis may expose errors in their work or may produce conclusions that contradict their own. However, these hypotheses have not previously been studied systematically.Methods and FindingsWe related the reluctance to share research data for reanalysis to 1148 statistically significant results reported in 49 papers published in two major psychology journals. We found the reluctance to share data to be associated with weaker evidence (against the null hypothesis of no effect) and a higher prevalence of apparent errors in the reporting of statistical results. The unwillingness to share data was particularly clear when reporting errors had a bearing on statistical significance.ConclusionsOur findings on the basis of psychological papers suggest that statistical results are particularly hard to verify when reanalysis is more likely to lead to contrasting conclusions. This highlights the importance of establishing mandatory data archiving policies.

Open Science: What, Why, and How

“…Estimates suggest that the majority of researchers do not adequately provide published data and materials, even when requested to do so by other researchers (Vanpaemel, Vermorgen, Deriemaecker, & Storms, 2015, 62%;Wicherts, Borsboom, Kats, & Molenaar, 2006, 73%). Though this lack of sharing may sometimes be due to purposeful withholding (see Wicherts, Bakker, & Molenaar, 2011), it is more likely that data and materials are simply stored improperly and lost (see Vines et al, 2014 (Deary et al, 2004). The old data have since been linked with current data, leading to dozens of new findings about how intelligence and other individual differences are associated with aging and long-term outcomes.…”

Do you speak open science? Resources and tips to learn the language

“…In 2015 Borgman identified four rationales for sharing research data: to reproduce research, to make those data that can be considered public assets, available to the public 4 , to leverage investments in research, and to advance research and innovation [18]. Several studies have furthermore reported that scientific papers accompanied by publicly available data are on average cited more often [19,20], and are moreover characterized by fewer statistical errors and a greater degree of robustness [21].…”

“…It is worth noting that this platform is not related to any specific journal or publisher, and as such constitutes a forum for public scientific discourse. Importantly, studies have shown that open peer-review can produce reviews of higher quality, with better verified claims, and more constructive criticisms, when compared to closed review [21,45]. Of course, one should keep in mind that open and transparent peer-review does not come without risks: especially young, early-career researchers might fear that by signing critical and thorough reviews they could become a target for retaliation at a sensitive point in their career.…”

Survey as a source of low quality research data

“…Sharing data is a natural in custom areas such as: astronomy and genomics [17]. Publishing the source data together with the research results, gives the proof of data good quality [18]. Authors of those publications choose their research to be publicly verified.…”