Open Science

Spellman, Barbara A.; Gilbert, Elizabeth; Corker, Katherine S.

doi:10.1002/9781119170174.epcn519

Cited by 42 publications

(41 citation statements)

References 134 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Open science is a broad term that encompasses a host of specific beliefs and behaviors (Spellman et al, 2018;Yarkoni, 2019). There is no agreed-upon definition of open science, and because of the variety of ideas that fall under the broad label, the lack of specificity means that having discussions about "diversity in open science" can be challenging.…”

Section: The Open Science Movementmentioning

confidence: 99%

Cultural Psychology, Diversity, and Representation in Open Science

Syed¹,

Kathawalla²

2020

Preprint

View full text Add to dashboard Cite

The current moment in psychology is one of great challenges and great opportunities. The open science movement--the move towards more transparent, credible, and reproducible science--has led to a redefinition of what constitutes “normal science.” However, the field of cultural psychology, broadly construed, has by and large not engaged with the open science movement, and likewise, the open science movement has by and large not engaged with cultural psychology. The purpose of the present chapter is to bring open science and cultural psychology closer together, highlighting how they can benefit one another. In doing so, we focus our discussion on three types of representations regarding diversity in psychological research and how they intersect with open science: representation of researchers, or the diversity of the scientists actually doing the research, representation of samples, or who is included as participants in our research studies, and representation of perspectives, or the substantive conceptual and theoretical views we bring to our work. For each of these three types of representation we outline the problem, and then discuss how embracing the principles and behaviors of open science can help.

show abstract

Section: The Open Science Movementmentioning

confidence: 99%

Cultural Psychology, Diversity, and Representation in Open Science

Syed¹,

Kathawalla²

2020

Preprint

View full text Add to dashboard Cite

show abstract

“…How meta-research fits into scientific practice. Meta-research and the broader "open science" movement (see Spellman, Gilbert, & Corker, 2018;and generally Dunleavy, 2020;Tennant et al, 2020aTennant et al, , 2020b, encourages the identification and correction of suboptimal research practices and the promotion of rigorous (i.e. severe) testing of hypotheses (Mayo, 2018).…”

Section: Solutions To the Replication Crisismentioning

confidence: 99%

Appraising Contemporary Social Work Research: Meta-Research on Statistical Reporting, Statistical Power, and Evidential Value

Dunleavy¹

2020

Preprint

View full text Add to dashboard Cite

Background. In recent years, the veracity of scientific findings has come under intense scrutinyin what has been called the “replication crisis” (sometimes called the “reproducibility crisis” or“crisis of confidence”). This crisis is marked by the propagation of scientific claims which weresubsequently contested, found to be exaggerated, or deemed false. The causes of this crisis aremany, but include poor research design, inappropriate statistical analysis, and the manipulationof study results. Though it is uncertain if social work is in the midst of a similar crisis, it is notunlikely, given parallels between the field and adjacent disciplines in crisis.Objective. This dissertation aims to articulate these problems, as well as foundational issues instatistical theory, in order to scrutinize statistical practice in social work research. In doing so, itparallels recent work in psychology, neuroscience, medicine, ecology, and other scientificdisciplines, while introducing a new program of meta-research to the social work profession.Method. Five leading social work journals were analyzed across a five-year period (2014-2018).In all 1,906 articles were reviewed, with 310 meeting inclusion criteria. The study was dividedinto three complementary parts. Statistical reporting practices were coded and analyzed in Part 1of the study (n = 310). Using reported sample sizes from these articles, a power survey wasperformed, in Part 2, for small, medium, and large effect sizes (n = 207). A novel statistical tool,the p-curve, was used in Part 3 to evaluate the evidential value of results from one journal(Research on Social Work Practice) and to assess for bias. Results from 39 of the 78 eligiblearticles were included in the analysis. Data and materials are available at: https://osf.io/45z3h/Results. Part 1: Notably, 86.1% of articles reviewed did not report an explicit alpha level. Apower analysis was performed in only 7.4% of articles. Use of p-values was common, beingreported in 96.8% of articles, but only 29% of articles reported them in exact form. Only 36.5%of articles reported confidence intervals; with the 95% coverage rate being the most common(reported in 31.3% of all studies). Effect sizes were explicitly reported in the results section ortables in a little more than half of articles (55.2%). Part 2: The mean statistical power for articleswas 57% for small effects, 88% for medium effects, and 95% for large effects. 61% of studiesdid not have adequate power (.80) to detect a small effect, 19% did not have adequate power todetect a medium effect, and 7% a large effect. A robustness test yielded similar but moreconservative estimates for these findings. Part 3: Both the primary p-curve and robustness testyielded right-skewed curves, indicating evidential value for the included set of results, and noevidence of bias.Conclusion. Overall, these findings provide a snapshot of the status of contemporary social workresearch. The results are preliminary but indicate areas where statistical design and reporting canbe improved in published research. The results of the power survey suggest that the field hasincreased mean statistical power compared to prior decades; though these findings are tentativeand have numerous limitations. The results of the p-curve demonstrate its potential as a tool forinvestigating bias within published research; while suggesting that the results included fromResearch on Social Work Practice have evidential value. In all this study provides a first steptowards a broader and more comprehensive assessment of the field.

show abstract

“…A core principle of open science is to improve access to findings from research, and this principle is driving countless researchers, funders, and policymakers to undertake a range of bottom-up and top-down initiatives to increase the accessibility of research findings (e.g., Marsden, in press;McKiernan et al, 2016;Spellman, Gilbert, & Corker, 2018). Language Learning now contributes to this agenda by asking authors of manuscripts accepted for publication to write short, nontechnical summaries of their article.…”

Section: Introductionmentioning

confidence: 99%

Extending the Reach of Research: Introducing Open Accessible Summaries at Language Learning

2019

View full text Add to dashboard Cite

Open science practices, which include efforts to enhance accessibility to data and materials and to improve the dissemination of findings to broader audiences, have been at the forefront of the open science movement worldwide. Language Learning is proud to extend its support of open science practices by participating in the OASIS (Open Accessible Summaries in Language Studies) initiative, which aims to make language‐related research openly available and easily accessible, both physically and conceptually. As part of the OASIS initiative, Language Learning now publishes accessible summaries of all accepted articles, including reviews and syntheses. Written in nontechnical language, accessible summaries provide information about each study's goals, its design and approach, and its results, highlighting findings that may be of interest to those outside academia, such as language educators. We outline key rationales leading to the development of accessible summaries, describe their core features, and discuss some of their strengths and weaknesses.

show abstract

Open Science

Cited by 42 publications

References 134 publications

Cultural Psychology, Diversity, and Representation in Open Science

Cultural Psychology, Diversity, and Representation in Open Science

Appraising Contemporary Social Work Research: Meta-Research on Statistical Reporting, Statistical Power, and Evidential Value

Extending the Reach of Research: Introducing Open Accessible Summaries at Language Learning

Contact Info

Product

Resources

About