Preregister if you want to.

Abstract

Prespecification of confirmatory hypothesis tests is a useful tool that makes our statistical tests informative. On the other hand, selectively reporting studies, measures, or statistical tests renders the probability of false positives higher than the p values would imply. The bad news is that it is usually difficult to tell how much higher the probability is. Fortunately, there are enormous opportunities to improve the quality of our science by preregistering our research plans. Preregistration is a highly distinctive strength that should increase our faith in the veracity and replicability of a research result.

Flexibility in reaction time analysis: many roads to a false positive?

“…The main idea behind preregistration consists of declaring a priori the specific procedures and the data analysis approach, including any preprocessing of RTs. In a preregistered study, all the researcher's degrees of freedom are squeezed into a predefined preprocessing and analysis pipeline [9,26,27], drawing a sharp distinction between confirmatory and exploratory analyses [28,29]. Different approaches to deal with distributional assumptions in preregistered research have been proposed [30].…”

Systematic Review and Meta-Analysis of Correlates of FFMQ Mindfulness Facets

“…Besides being interesting in itself, studying correlations has the additional benefit of leading to a large sample size, which—as mentioned above—is important for the chosen method to study the impact of possible publication bias. Given another important recent concern, namely research standards in psychology (e.g., Bones, 2012; Brown et al, 2013; Open Science Collaboration, 2015; Ledgerwood, 2016), another aspect that is taken into account is adherence to (two aspects of) good research practice (Finkel et al, 2015): preregistration (Moore, 2016) and transparency; the latter will be proxied by availability (whether the underlying data used and/or the paper are openly available) and readability (whether the abstract is structured or not). Partial correlations with outcomes will also be studied, as the unique contributions of facets seem interesting both in themselves (above it was argued that for some facets positive effects might seem counter-intuitive, thus it seems important to exhibit to what extent they uniquely contribute to positive outcomes) and in the design of mindfulness trainings and interventions (by indicating which facets should be preferentially strengthened).…”

Construct Validation in Social and Personality Research

“…To know it thoroughly involves knowing its quantity as well as its quality.—Edward ThorndikeThe dependability of research findings from diverse fields has recently come under scrutiny, including psychology (Pashler & Wagenmakers, 2012; Sijtsma, 2016; Simmons, Nelson, & Simonsohn, 2011). Such concerns have sparked discussion and facilitated the reexamination of many core statistical and methodological practices which might have contributed to a “replication crisis.” These include the role of null hypothesis significance testing, the reporting of effect sizes and their confidence intervals (Cumming, 2014; Wilkinson & Task Force on Statisical Inference, 1999), data sharing and conducting replications (Nosek et al, 2015; Open Science Collaboration, 2015), and the preregistration of studies (Moore, 2016). To this point in discussions, measurement has gone relatively unexamined.…”