Badges to Acknowledge Open Practices: A Simple, Low-Cost, Effective Method for Increasing Transparency

Abstract

Data Availability Statement: All data and materials are publicly accessible at https://osf.io/rfgdw/. Also, we preregistered our study design and analysis plan. You can find the preregistration at https://osf.io/ipkea/.Funding: The authors received no specific funding for this work.Competing Interests: Brian Nosek created the badges to acknowledge open practices, and Brian Nosek and Mallory Kidwell are on a committee maintaining the badges. The badges and specifications for earning them are CC0 licensed with and in four comparison journals. We report an increase in reported data sharing of more than an order of magnitude from baseline in Psychological Science, as well as an increase in reported materials sharing, although to a weaker degree. Moreover, we show that reportedly available data and materials were more accessible, correct, usable, and complete when badges were earned. We demonstrate that badges are effective incentives that improve the openness, accessibility, and persistence of data and materials that underlie scientific research.

Practical tools and strategies for researchers to increase replicability

“…42 Several journals now require or reward open data (with extra recognition for papers that share their data), and these policies are strongly related to increased data availability. 43,44 Ideally, when data are shared, there should be a quality check to ensure they are relevant, complete, and usable. 44,45 One way to increase the likelihood that data remain available over time is to publish them in online data repositories.…”

“…43,44 Ideally, when data are shared, there should be a quality check to ensure they are relevant, complete, and usable. 44,45 One way to increase the likelihood that data remain available over time is to publish them in online data repositories. An example of such a repository is the Open Science Framework.…”

The Research Reproducibility Crisis and Economics of Science

“…A series of reforms was accordingly implemented, with the most important ones being the imposition of reporting guidelines, open data, and preregistration at the journal and funder level (De Angelis et al, 2004;Simer et al, 2010;Collins and Tabak, 2014;Eich, 2014;McNutt, 2014). As far as we know, rigorous policy assessment did not take place, although ex post studies have found some evidence of success (Kane et al, 2007;Kaplan and Irvin, 2015;Kidwell et al, 2016).…”

Is useful research data usually shared? An investigation of genome-wide association study summary statistics

“…Descriptions of the exact nature of the data available would not only help automation but also researchers scanning multiple articles to find relevant data for a new study. Thus, if more journals required data sharing statements and employed guidelines to ensure that the shared data was described in detail, or provided virtual rewards [50] for these activities, then this would support the level of automated data discovery that would be necessary to monitor data sharing and systematically identify shared data for later re-use.…”