Bayesian reanalysis of null results reported in medicine: Strong yet variable evidence for the absence of treatment effects

Hoekstra, Rink; Monden, Rei; Ravenzwaaij, Don van; Wagenmakers, Eric‐Jan

doi:10.1371/journal.pone.0195474

Cited by 43 publications

(56 citation statements)

References 30 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Besides providing p-values to express the rejection of a null hypothesis, extra information is provided by the Bayes factor (BF) about the strength of the evidence in favor of the alternative hypothesis over the null hypothesis (BF 10 ) or vice versa [44,45,46].…”

Section: Methodsmentioning

confidence: 99%

The Impact of Childhood Maltreatment on Intravenous Ketamine Outcomes for Adult Patients with Treatment-Resistant Depression

et al. 2019

View full text Add to dashboard Cite

Childhood maltreatment is associated with a poor treatment response to conventional antidepressants and increased risk for treatment-resistant depression (TRD). The N-methyl-D-aspartate receptor (NDMAR) antagonist ketamine has been shown to rapidly improve symptoms of depression in patients with TRD. It is unknown if childhood maltreatment could influence ketamine’s treatment response. We examined the relationship between childhood maltreatment using the Childhood Trauma Questionnaire (CTQ) and treatment response using the Quick Inventory of Depressive Symptoms–Self Report (QIDS-SR) in TRD patients receiving intravenous ketamine at a community outpatient clinic. We evaluated treatment response after a single infusion (n = 115) and a course of repeated infusions (n = 63). Repeated measures general linear models and Bayes factor (BF) showed significant decreases in QIDS-SR after the first and second infusions, which plateaued after the third infusion. Clinically significant childhood sexual abuse, physical abuse, and cumulative clinically significant maltreatment on multiple domains (maltreatment load) were associated with better treatment response to a single and repeated infusions. After repeated infusions, higher load was also associated with a higher remission rate. In contrast to conventional antidepressants, ketamine could be more effective in TRD patients with more childhood trauma burden, perhaps due to ketamine’s proposed ability to block trauma-associated behavioral sensitization.

show abstract

Section: Methodsmentioning

confidence: 99%

The Impact of Childhood Maltreatment on Intravenous Ketamine Outcomes for Adult Patients with Treatment-Resistant Depression

et al. 2019

View full text Add to dashboard Cite

show abstract

“…In comparison, a Bayes factor of BF 10 = 1/5 (or the inverse of 5), means the observed data are five times more likely to have occurred under the null hypothesis than under the alternative hypothesis. 1 The application of the JZS Bayes factor for a large-scale reanalysis of published results is not without precedent (Hoekstra, Monden, van Ravenzwaaij, & Wagenmakers, 2018). We build upon the work of Hoekstra and colleagues in taking the results of such a Bayesian reanalysis as a starting point for selecting replication targets (for a similar approach, see Pittelkow, Hoekstra, & van Ravenzwaaij, 2019).…”

Section: Statistical Considerationsmentioning

confidence: 99%

When and Why to Replicate: As Easy as 1, 2, 3?

Field

Hoekstra

Bringmann

et al. 2019

Collabra: Psychology

Self Cite

View full text Add to dashboard Cite

The crisis of confidence in psychology has prompted vigorous and persistent debate in the scientific community concerning the veracity of the findings of psychological experiments. This discussion has led to changes in psychology's approach to research, and several new initiatives have been developed, many with the aim of improving our findings. One key advancement is the marked increase in the number of replication studies conducted. We argue that while it is important to conduct replications as part of regular research protocol, it is neither efficient nor useful to replicate results at random. We recommend adopting a methodical approach toward the selection of replication targets to maximize the impact of the outcomes of those replications, and minimize waste of scarce resources. In the current study, we demonstrate how a Bayesian re-analysis of existing research findings followed by a simple qualitative assessment process can drive the selection of the best candidate article for replication.

show abstract

“…The nullhypothesis-significance-testing procedure was complemented with the estimation of Bayes factors for null results, as delivered by JASP (JASP Team, 2020). Bayes factors for null results [null/alternative]) indicate the likelihood of observing the null data under the null hypothesis compared to the alternative hypothesis, thus quantifying the strength of evidence in favor of absent effects (Hoekstra et al, 2018;Wagenmakers et al, 2018). A rule of thumb is to consider a Bayes factor between 1 and 3 as weak evidence, though favoring the null hypothesis, between 3 and 10 as substantial, between 10 and 30 strong, and between 30 and 100 as decisive evidence (Jarosz and Wiley, 2014).…”

Section: Discussionmentioning

confidence: 99%

The Mozart Effect on the Episodic Memory of Healthy Adults Is Null, but Low-Functioning Older Adults May Be an Exception

2020

View full text Add to dashboard Cite

Literature on the effects of passive music listening on cognitive performance is mixed, showing negative, null or positive results depending on cognitive domain, age group, temporal relation between music and task (background music vs. music before task, the latter known as Mozart effect), or listener-dependent variables such as musical preference. Positive effects of background music on the two components of episodic memory-item and source memory-for verbal materials seem robust and ageindependent, and thus deserve further attention. In the current study, we investigated two potential enhancers of music effects on episodic memory: stopping music before task performance (Mozart effect) to eliminate music-related distraction and using preferred music to maximize reward. We ran a main study on a sample of 51 healthy younger adults, along with a pilot study with 12 older adults, divided into low-vs. high functioning according to cognitive performance in a screening test. Against our expectations, Bayesian analyses showed strong evidence that music had no advantage over silence or environmental sounds in younger adults. Preferred music had no advantage either, consistent with the possibility that music-related reward had no impact on episodic memory. Among older adults, low-but not high-functioning participants' item memory was improved by music-especially by non-preferred music-compared to silence. Our findings suggest that, in healthy adults, prior-to-task music may be less effective than background music in episodic memory enhancement despite decreased distraction, possibly because reward becomes irrelevant when music is stopped before the task begins. Our pilot findings on older adults raise the hypothesis that lowfunctioning older participants relate to prior-to-task auditory stimulation in deviant ways when it comes to episodic memory enhancement.

show abstract

Bayesian reanalysis of null results reported in medicine: Strong yet variable evidence for the absence of treatment effects

Cited by 43 publications

References 30 publications

The Impact of Childhood Maltreatment on Intravenous Ketamine Outcomes for Adult Patients with Treatment-Resistant Depression

The Impact of Childhood Maltreatment on Intravenous Ketamine Outcomes for Adult Patients with Treatment-Resistant Depression

When and Why to Replicate: As Easy as 1, 2, 3?

The Mozart Effect on the Episodic Memory of Healthy Adults Is Null, but Low-Functioning Older Adults May Be an Exception

Contact Info

Product

Resources

About