Outcome Reporting Bias in Randomized Experiments on Substance Use Disorders

Questionable Research Practices and Open Science in Quantitative Criminology

“…Beginning with concern, the high rate of QRP usage is disappointing because QRPs contribute to false and misleading findings. Our evidence is consistent with the conclusion that many findings in criminology are likely false positives (Sweeten, 2020;Gelman, Skardhamar, & Aaltonen, 2020;Wooditch et al, 2020). Existing efforts to make criminologists aware of the pitfalls of QRPs (see Burt, 2020) should be strengthened.…”

“…Thus, the same career-oriented criminologists may also want to cash in on the reputational benefits of using OSPs when they can (e.g., when findings are initially significant). The evidence that criminologists deviate selectively from their preregistrations to increase effect sizes is consistent with this selective transparency explanation (Wooditch et al, 2020).…”

“…We suggest future studies should use a more specific and detailed definition of preregistration (e.g., specifying that the plan should be recorded in a timestamped repository). Subsequent research should also ask whether criminologists adhere to their preregistrations, which we did not ask about; some research suggests they often do not (Wooditch et al, 2020).…”

“…Additionally, there is a sizable inverse relationship between sample size and effect size in criminology (Nelson, Wooditch, & Dario, 2015), which is a telltale sign that something is amiss (Gelman, Skardhamar, & Aaltonen, 2020;Levine, Asada, & Carpenter, 2009). As importantly, published experiments in criminology often differ from the plan described at the proposal stage, and the more they differ the larger the published effect size, suggesting criminologists deviate selectively from research protocols in a way that exaggerates results (Wooditch et al, 2020). Despite this suggestive evidence, however, there have been no surveys on QRP use in criminology, as there have been in other disciplines.…”

Are Most Published Criminological Research Findings Wrong? Taking Stock of Criminological Research Using a Bayesian Simulation Approach

“…The pervasiveness of replication issues in scientific fields that have looked for them suggests these issues likely extend to other scientific fields that have yet to engage in the same undertaking. In criminology, recent work has provided preliminary evidence suggesting many of the factors responsible for false-positives rates in other fields are also present in criminology (Barnes et al, 2020;Chin, 2021;McNeeley & Warner, 2015;Pridemore et al, 2018;West et al, 2020;Wooditch, Fisher, et al, 2020;Wooditch, Sloas, et al, 2020).…”