Preregistration of secondary data analysis: A template and tutorial

Abstract

Preregistration has been lauded as one of the solutions to the so-called ‘crisis of confidence’ in the social sciences and has therefore gained popularity in recent years. However, the current guidelines for preregistration have been developed primarily for studies where new data will be collected. Yet, preregistering secondary data analyses--- where new analyses are proposed for existing data---is just as important, given that researchers’ hypotheses and analyses may be biased by their prior knowledge of the data. The need for proper guidance in this area is especially desirable now that data is increasingly shared publicly. In this tutorial, we present a template specifically designed for the preregistration of secondary data analyses and provide comments and a worked example that may help with using the template effectively. Through this illustration, we show that completing such a template is feasible, helps limit researcher degrees of freedom, and may make researchers more deliberate in their data selection and analysis efforts.

Research practices for a robust psychological science of adult development and aging.

“…Finally, with the growing use of large existing dataset sets, the implementation of preregistration with secondary data analysis raises some thorny issues (e.g., posing a priori hypotheses given familiarity with a dataset) with which we are still coming to terms (e.g., van den Akker et al, 2021; Scott & Kline, 2019; Weston et al, 2019). We certainly encourage the use of advances in guidelines and preregistration templates for secondary data analysis going forward.…”

Using preregistration as a tool for transparent fNIRS study design

“…Researchers should consider the possible scope of their study: ideally, the study should tell a specific story; otherwise, if the scope goes beyond a single story, it will likely require defining a set of experiments or even a whole research program on this topic. Importantly, besides original empirical and replication studies, researchers can also preregister methodological studies (e.g., comparison of different motion artifact correction algorithms), exploratory studies 44 , or secondary analyses of existing data sets 37,54 .…”

“…To make the task of writing a preregistration protocol more tractable, proponents of open science have developed guides, checklists, and templates to facilitate transparent study design, preregistration, and reporting for various types of research including neuroimaging 7,[36][37][38][39][40][41][42] . For fNIRS research, however, no similar resources exist yet and would be highly desired 17 .…”

Preregistration: Definition, Advantages, Disadvantages, and How It Can Help Against Questionable Research Practices

“…Above, we described common characteristics of current preregistration templates (e.g., Crüwell & Evans, 2019;Kirtley, Lafit, Achterhof, Hiekkaranta, & Germeys, 2020;Krypotos et al, 2019;Van den Akker et al, 2019;Veer & Giner-Sorolla, 2016). As mentioned previously, however, different types of studies need different preregistration elements, and more templates are being introduced 4.0.6 Remaining sections..…”