Appreciating the Significance of Non-significant Findings in Psychology

Abstract

Hypothesis tests for which the null hypothesis cannot be rejected ("null findings") are often seen as negative outcomes in psychology. Null findings can, however, bear important insights about the validity of theories and hypotheses. In addition, the tendency to publish mainly significant findings is considered a key reason for failures to replicate previous studies in various fields, including psychology. In this editorial, we discuss the relevance of non-significant results in psychological research and ways to render these results more informative. We discuss the possibility to test whether null findings provide evidence for the absence or negligible size of an effect, based both on frequentist and Bayesian statistical methods. We further discuss the role of adequate power analysis in obtaining informative evidence for null findings, with a special emphasis on student research. Lastly, we encourage researchers at all career stages to submit null findings for publication.

Graded fMRI Neurofeedback Training of Motor Imagery in Middle Cerebral Artery Stroke Patients: A Preregistered Proof-of-Concept Study

“…This stepwise Bayesian approach provides us with more flexibility to detect larger effects and hence to stop sampling earlier. It also allows flexible accumulation of evidence for the null hypothesis, which presents one main advantage of Bayesian hypothesis testing compared to conventional frequentist hypothesis tests (Mehler et al, 2019a ). To control for early stopping due to false positives or false negatives, we have set a relatively conservative stopping threshold of BF = 10 for both the alternative and the null hypothesis.…”

Neurofeedback training in major depressive disorder: A systematic review of clinical efficacy, study quality and reporting practices

“…Further, alternative sampling strategies such as sequential Bayes Factor (SBF) sampling may be worthwhile exploring for clinical neurofeedback studies (Schönbrodt and Wagenmakers, 2018). Lastly, we recommend that null findings are followed up with appropriate statistical tests that allow providing evidence for the absence of an effect (Mehler et al, 2019).…”

The potential of functional near-infrared spectroscopy-based neurofeedback – a systematic review and recommendations for best practice

“…In general, the field will benefit from adopting more rigorous research and reporting practices as encouraged by a recent consensus (Aczel et al, 2019;Ros et al, 2020) to improve the likelihood of replicability and reproducibility (Mehler, 2019). These measures include sampling plans that are ideally based on adequate power analyses which render both positive and negative findings more informative (Mehler et al, 2019). We acknowledge that it remains challenging to define the smallest effect sizes of interest, which are context-specific.…”