On the Lack of Consensus over the Meaning of Openness: An Empirical Study

Abstract

This study set out to explore the views and motivations of those involved in a number of recent and current advocacy efforts (such as open science, computational provenance, and reproducible research) aimed at making science and scientific artifacts accessible to a wider audience. Using a exploratory approach, the study tested whether a consensus exists among advocates of these initiatives about the key concepts, exploring the meanings that scientists attach to the various mechanisms for sharing their work, and the social context in which this takes place. The study used a purposive sampling strategy to target scientists who have been active participants in these advocacy efforts, and an open-ended questionnaire to collect detailed opinions on the topics of reproducibility, credibility, scooping, data sharing, results sharing, and the effectiveness of the peer review process. We found evidence of a lack of agreement on the meaning of key terminology, and a lack of consensus on some of the broader goals of these advocacy efforts. These results can be explained through a closer examination of the divergent goals and approaches adopted by different advocacy efforts. We suggest that the scientific community could benefit from a broader discussion of what it means to make scientific research more accessible and how this might best be achieved.

“A good collaboration is based on unique contributions from each side”: assessing the dynamics of collaboration in stem cell science

“…As with previous studies (Grubb and Easterbrook 2011), the number of respondents who mentioned being scooped as a concern was higher than those who recounted actual prior experience of being scooped. While being scooped is an unpleasant and irritating experience for a senior academic, it is potentially more damaging, and thus a particular worry, for more junior scientists:

“[A]s an academic I have a non-permanent contract.

…”

BRICK v0.2, a simple, accessible, and transparent model framework for climate and regional sea-level projections

“…In an effort to be truly open source and freely available, all codes -including the physical model, statistical model, and processing and plotting scripts used for the results shown here -are available through a download server as well as the Github repository provided in the Code Availability section of this article. Providing all code and data necessary to recreate this study is a critical component of reproducible research (MurrayRust and Murray-Rust, 2014) and can help to build trust between the general public and the scientific community (Easterbrook, 2014;Grubb and Easterbrook, 2011).…”

“…Frequent and thorough code review by other team members as well as outside agents is another critical step towards good scientific coding practices (Wilson et al, 2014), and "peer review needs to be supplemented with a number of other mechanisms that help to establish the correctness and credibility of scientific research" (Grubb and Easterbrook, 2011). Wilson et al (2014) also note that a number of high-profile research articles have been retracted or revised because of errors in the code.…”

Open science

“…Indeed, although some writers view open science through an economic lens [Willinsky, 2005;Cribb and Sari, 2010], the open science community concentrates on the importance of open access to data [Murray-Rust et al, 2010] Open science, as a practice and concept, is still in an experimental period of rapid evolution and diversification. Grubb and Easterbrook [2011], in a small-scale qualitative survey, showed that even among scientists who identified themselves as advocates of openness, there was a low degree of consensus on precisely what it comprised. This diversity may arise from a variety of causes; the different circumstances in which people work, for example in a large research group or alone, or publicly or privately funded.…”