“Like Upgrading From a Typewriter to a Computer”: Registered Reports in Education Research

Abstract

Registered reports engender a review process in which reviewers evaluate the merits of the research question rather than the magnitude or direction of the results. In this process, authors first submit introduction, methods, and prespecified analyses sections of an article. Next, an initial peer-review process assesses the merits of the research question, in the absence of knowing the results. Worthy articles are granted an “in-principle acceptance.” The authors, then conduct their study, analyze their data, write up the remainder of their article, and resubmit it. The in-principle acceptance means that if the authors adhere to the research design as originally submitted, then the article will be accepted. Building from seven of the earliest registered reports in education research, our introduction describes the benefits, opportunities, and challenges of registered reports as a means to increasing transparency in research and reducing the number of illusory results.

Registered Reports in Special Education: Introduction to the Special Series

“…One approach for editors is to do a trial run by conducting a special issue or series, which allows them to experience and develop procedures for registered reports REGISTERED REPORTS 19 without committing their journal to the practice. For instance, AERA Open adopted registered reports after the positive experience of doing a special topics series on the practice (Reich et al, 2020). One potential concern for editors is the level of methodological detail required in stage-1 manuscripts may result in stage-2 manuscripts exceeding journal page limits.…”

“…Among the listed journals that accept registered reports as a regular submission option are a few in the fields of education (i.e., British Journal of Educational Psychology; Frontiers in Education; Journal of Research in Reading; Learning and Instruction; Mind, Brain, and Education ; and Scientific Studies of Reading ) and special education/exceptionality (i.e., Exceptional Children, Gifted Child Quarterly ). Although not included on the curated list, we note AERA Open has recently begun to accept registered reports as regular submissions (Reich et al, 2020).…”

“…Journals related to education include AERA Open, Child Development , and Learning Disability Quarterly . A special topics series on registered reports in AERA Open (Reich et al, 2020) included seven studies:…”

Implications of the Open Science Era for Educational Psychology Research Syntheses

“…Moreover, research syntheses may compound the problem to the extent that they too often suffer from parallel biases that call their validity into question. Luckily, scholars have begun to generate strategies (e.g., preregistration, open and archived data, registered reports, prospective meta-analysis) to change our current research norms in the social and health sciences (e.g., Berlin & Ghersi, 2005;Cook et al, 2018;Nosek et al, 2015;Reich et al, 2020;Simmons et al, 2011). These solutions not only have implications for the primary research they are meant to guide, but for the research syntheses that build off primary research.…”

“…Registered reporting is yet another way that dissemination biases and the questionable practices that influence the outcomes of primary research and research syntheses alike can be addressed. Registered reports entail that research is accepted for publication via peer review based on an evaluation of pre-registered study plans prior to the study actually being conducted (Chambers et al, 2014;Reich et al, 2020). Once the research is conducted, publication is contingent only on the extent to which data plans and analyses were adhered to and exploratory analyses are clearly identified and justified.…”