Open Science Practices are on the Rise: The State of Social Science (3S) Survey

Abstract

Has there been meaningful movement toward open science practices within the social sciences in recent years? Discussions about changes in practices such as posting data and pre-registering analyses have been marked by controversy—including controversy over the extent to which change has taken place. This study, based on the State of Social Science (3S) Survey, provides the first comprehensive assessment of awareness of, attitudes towards, perceived norms regarding, and adoption of open science practices within a broadly representative sample of scholars from four major social science disciplines: economics, political science, psychology, and sociology. We observe a steep increase in adoption: as of 2017, over 80% of scholars had used at least one such practice, rising from one quarter a decade earlier. Attitudes toward research transparency are on average similar between older and younger scholars, but the paceof change differs by field and methodology. According with theories of normal science and scientific change, the timing of increases in adoption coincides with technological innovations and institutional policies. Patterns are consistent with most scholars underestimating the trend toward open science in their discipline.

Practice what you preach: Credibility-enhancing displays and the growth of open science

“…and both their correlation and levels vary across fields (e.g. Christensen et al, 2020). In sum, OSPs appear to have low descriptive normativity and there is also widespread approval of open science, suggesting injunctive normativity and/or independent preferences to engage in OSPs.…”

“…Houtkoop et al, 2018). Encouragingly, in perhaps the most sophisticated and among the most recent self-report studies of N=2,801 social scientists (21.3% in psychology) in 2018, an estimated upper bound of 73% reported ever having publicly posted their data (although only 51% among psychologists; Christensen et al, 2020). Still, these trends are far from representative across the field as a whole, particularly in cross-sectional analysis; for example, in a contemporary audit study of 250 psychology articles published between 2014 and 2017, only 2% had open data (Hardwicke et al, 2020).…”

“…Still, these trends are far from representative across the field as a whole, particularly in cross-sectional analysis; for example, in a contemporary audit study of 250 psychology articles published between 2014 and 2017, only 2% had open data (Hardwicke et al, 2020). These trends are similar for other OSPs: while 60% of psychologists report ever having posted open materials and 44% of psychologists report ever having preregistered an experiment (Christensen et al, 2020), of papers audited, only 14% had open materials and only 3% of papers audited had preregistered experiments (Hardwicke et al, 2020).…”

NSF 19-609 Proposal: Evaluating Rigor and Transparency Culture Change Initiatives

“…This is particularly important to establish shared understanding across the variety of sub fields that will be included in this study. Except for the replication prompt, these are adapted from Christensen et al (2019).…”

Can we Trust Science and, if so, Why?

“…While the intensity differs across academic fields, it is hard to find a discipline in which the systemic lack in replicability and transparency is not being discussed (Baker 2016;Christensen et al 2019). evidence on these topics is still scarce, it is not true that recent meta-scientific research has only upset singular studies, but it has also fundamentally challenged textbook phenomena that previously had widespread support in the literature such as ego depletion (Friese et al 2019) or social priming (Chivers 2019).…”