Reform of statistical inference in psychology: The case ofMemory &amp; Cognition

Finch, Sue; Cumming, Geoff; Williams, J.; Palmer, Lee; Griffith, Elvira; Alders, C.M.; Anderson, James L.; Goodman, Olivia

doi:10.3758/bf03195577

Cited by 42 publications

(36 citation statements)

References 30 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Despite widespread agreement that current practices are flawed, some disciplines have shown remarkable resistance to change (e.g., psychology and economics; Altman 2004), as measured by reporting practices in journals. After almost half a century of criticisms, and in the case of psychology, serious editorial and institutional intervention from professional bodies such as the American Psychological Association, statistical‐reporting practices in journals remain largely unmoved and heavily dependent on NHST‐based procedures (Finch et al 2004; Fidler et al 2005).…”

Section: Introductionmentioning

confidence: 99%

Impact of Criticism of Null‐Hypothesis Significance Testing on Statistical Reporting Practices in Conservation Biology

Fidler

Burgman

Cumming

et al. 2006

Conservation Biology

Self Cite

127

108

View full text Add to dashboard Cite

Over the last decade, criticisms of null-hypothesis significance testing have grown dramatically, and several alternative practices, such as confidence intervals, information theoretic, and Bayesian methods, have been advocated. Have these calls for change had an impact on the statistical reporting practices in conservation biology? In 2000 and 2001, 92% of sampled articles in Conservation Biology and Biological Conservation reported results of null-hypothesis tests. In 2005 this figure dropped to 78%. There were corresponding increases in the use of confidence intervals, information theoretic, and Bayesian techniques. Of those articles reporting null-hypothesis testing--which still easily constitute the majority--very few report statistical power (8%) and many misinterpret statistical nonsignificance as evidence for no effect (63%). Overall, results of our survey show some improvements in statistical practice, but further efforts are clearly required to move the discipline toward improved practices.

show abstract

Section: Introductionmentioning

confidence: 99%

Impact of Criticism of Null‐Hypothesis Significance Testing on Statistical Reporting Practices in Conservation Biology

Fidler

Burgman

Cumming

et al. 2006

Conservation Biology

Self Cite

127

108

View full text Add to dashboard Cite

show abstract

“…Even so, psychologists and other social scientists have been slow to include CIs in their research reports. Persistence by Geoffrey Loftus raised the percentage of articles that reported CIs in Memory & Cognition from 8 % to 45 % during his tenure as editor, but this fell to 27 % after his term ended (Finch et al, 2004). Finch et al estimated that only about 22 % of published experimental psychological research includes CIs.…”

mentioning

confidence: 99%

BootES: An R package for bootstrap confidence intervals on effect sizes

2013

View full text Add to dashboard Cite

Bootstrap Effect Sizes (bootES; Gerlanc & Kirby, 2012) is a free, open-source software package for R (R Development Core Team, 2012), which is a language and environment for statistical computing. BootES computes both unstandardized and standardized effect sizes (such as Cohen's d, Hedges's g, and Pearson's r) and makes easily available for the first time the computation of their bootstrap confidence intervals (CIs). In this article, we illustrate how to use bootES to find effect sizes for contrasts in between-subjects, withinsubjects, and mixed factorial designs and to find bootstrap CIs for correlations and differences between correlations. An appendix gives a brief introduction to R that will allow readers to use bootES without having prior knowledge of R.

show abstract

“…For example, experts in clinical trials methodology developed the CONSORT guidelines to improve the reporting of randomized control trials , which was simultaneously published in dozens of medical IMPLEMENTATION SCIENCE OF QUANTITATIVE METHODS 17 journals. The APA Publication manual provides standards for reporting many basic statistics (American Psychological Association, 2010), and editors and editorial boards have tried to shape how authors report studies, such as banning or requiring more stringent p-values (Finch et al, 2004;Ioannidis, 2018), or implementing incentives to promote specific practices such as data or materials sharing (Kidwell et al, 2016).…”

Section: How Can We Improve the Application Of Quantitative Methods Imentioning

confidence: 99%

Using implementation science to close the gap between the optimal and typical practice of quantitative methods in clinical science

King¹,

Pullmann²,

Lyon³

et al. 2018

Preprint

View full text Add to dashboard Cite

Quantitative methods remain the fundamental approach for hypothesis testing, but in approaches to data analysis there is substantial evidence of a gap between what is optimal and what is typical. It is clear that diffusion and dissemination alone are not maximally effective at improving data analytic practices in clinical psychological science. Amid declines in quantitative psychology training, and growing demand for advanced quantitative methods, applied researchers are increasingly called upon to conduct and evaluate research using methods in which they lack expertise. This "research-to-practice" gap in which rigorously developed and empirically supported quantitative methods are not applied in practice has received little attention. In this article, we describe how implementation science, which aims to reduce the research-to-practice gap in health care, offers a promising set of methods for closing the gap for quantitative methods. By identifying determinants of practice (i.e., barriers and facilitators of change), implementation strategies can be selected to increase adoption and high-fidelity application of new quantitative methods to improve scientific inferences and policy and practice decisions in clinical psychological science. General scientific summary:Making studies more replicable will require more effective use of statistics in research, but there is a large gap between how statistics are applied in psychological research and how they should be applied. The current article describes how the lessons of clinical implementation science, which has focused on getting evidence based treatments into community practice settings, may be applied to improve research in clinical psychology.

show abstract

Reform of statistical inference in psychology: The case ofMemory & Cognition

Cited by 42 publications

References 30 publications

Impact of Criticism of Null‐Hypothesis Significance Testing on Statistical Reporting Practices in Conservation Biology

Impact of Criticism of Null‐Hypothesis Significance Testing on Statistical Reporting Practices in Conservation Biology

BootES: An R package for bootstrap confidence intervals on effect sizes

Using implementation science to close the gap between the optimal and typical practice of quantitative methods in clinical science

Contact Info

Product

Resources

About