Reproducible research practices and transparency across linguistics

Abstract

Scientific studies of language span across many disciplines and provide evidence for social,  cultural, cognitive, technological, and biomedical studies of human nature and behavior. As it becomes increasingly empirical and quantitative, linguistics has been facing challenges and limitations of the scientific practices that pose barriers to reproducibility and replicability. One of the  proposed solutions to the widely acknowledged reproducibility and replicability crisis has been the implementation of transparency practices,  e.g., open access publishing, preregistrations, sharing study materials, data, and analyses, performing study replications, and declaring conflicts of interest. Here, we have assessed the prevalence of these practices in 600 randomly sampled journal articles from linguistics across two time points. In line with similar studies in other disciplines, we found that 35% of the articles were published open access and the rates of sharing materials, data, and protocols were below 10%. None of the articles reported preregistrations, 1% reported replications, and 10% had conflict of interest statements. These rates have not increased noticeably between 2008/2009 and 2018/2019, pointing to remaining barriers and the slow adoption of open and reproducible research practices in linguistics. To facilitate adoption of these practices, we provide a range of recommendations and solutions for implementing transparency and improving reproducibility of research in linguistics.

Opening open science to all: Demystifying reproducibility and transparency practices in linguistic research

“…Importantly, statements of conflict of interest are underused in linguistic research. Bochynska et al (2023) surveyed open and transparent practices in linguistics and found that only 10% of the articles sampled included statements of conflict of interest, and, among those 10%, none declared any conflicts (See also Cristea & Ioannidis, 2018;Hardwicke et al, 2022Hardwicke et al, , 2020. Positionality statements are likely even less common.…”

“…When submitting a study for publication, the positionality statement can be included in additional materials if the word limit is a concern. For examples of positionality statements in linguistic research, the interested reader is directed to Bochynska et al (2023) and Weissler et al (2023).…”

“…Learning, Language and Speech), and, in some cases, have made data sharing a requirement for publication (e.g., Applied Psycholinguistics). Nonetheless, open data is still the exception rather than the norm in linguistics (Bochynska et al, 2023). In this section, we provide more detail Open data are particularly important for the field of linguistics, for all of the aforementioned reasons, and also because linguistics is Western, Educated, Industrialized, Rich, and Democratic (WEIRD, Bochynska et al, 2023).…”

“…Nonetheless, open data is still the exception rather than the norm in linguistics (Bochynska et al, 2023). In this section, we provide more detail Open data are particularly important for the field of linguistics, for all of the aforementioned reasons, and also because linguistics is Western, Educated, Industrialized, Rich, and Democratic (WEIRD, Bochynska et al, 2023). That is to say, the majority of linguistic research is concentrated on specific language, mainly Indo-Germanic, in overrepresented communities, by privileged scholars.…”

“…The resulting methodological framework and associated techniques have reshaped research methods in Psychology, and, slowly but surely, are making their way into related fields. While many agree that open science practices represent a positive step forward in improving scientific rigor, these practices, by and large, have not been adopted in the field of linguistics (Bochynska et al, 2023). One reason for the slow adoption in linguistics may be related to the fact that engaging in open science is no trivial feat.…”