Promoting computationally reproducible research in applied linguistics: Recommended practices and considerations

Estimating reliability for response-time difference measures: Toward a standardized, model-based approach

“…Moreover, we join previous calls for the sharing of reproducible analysis code (Hui & Huntley, 2021;In'nami et al, 2022). Analysis code can reveal the procedure one has undertaken to arrive at the reported reliability.…”

The future of replication in applied linguistics: Toward a standard for replication studies

“…However, other than some discussion about the importance, value, and ethics of replications, most of these texts provide very little guidance about how to conduct and report replication studies. Hatch and Farhady's (1982) text Research design and statistics for applied linguistics, for example, provides substantive discussion and guidance about approaches to research design, data analysis, and reporting empirical research but includes only passing remarks about replication research (for similar treatments, see D ö rnyei, 2007;Plonsky, 2015b). Although some research methodology textbooks discuss the need and value of doing replication studies, prospective replication researchers are often provided with little-to-no guidance about conducting or reporting replications (e.g., Mackey & Gass, 2016;Rose et al, 2020).…”

Open science: Considerations and issues for TESOL research

“…Although TESOL and applied linguistics data more broadly are arguably rarely as sensitive as those in clinical trials, openly sharing data, if done without due caution, may violate the privacy of participants and other members of their respective communities. To help protect participants' data, at a minimum, the following steps should be taken: (a) conducting OS training sessions for data holders (Hicks, 2023; see also Hui & Huntley, in press for a multi‐tiered approach to incorporating OS training in applied linguistics graduate programs), (b) publishing field‐specific decision trees on sharing research data (e.g., Zipper et al., 2019, for water science research), (c) submitting data management plans with the manuscripts (e.g., “a written data privacy and security statement as part of the submission process;” Zipper et al., 2019, p. 5207), (d) discussing with participants how their data will be shared and ensuring that they can make an informed decision even if the purpose of the study is unknown (Dennis et al., 2019), (e) using controlled or managed access for sensitive data that can be reidentified (Dyke et al., 2015; Maritsch et al., 2022), and (f) sharing simulated data if the original data cannot be shared (In'nami, Mizumoto, Plonsky, & Koizumi, 2022). The latter is a viable option if the data are being collected from vulnerable groups (e.g., minorities and children).…”