The effects of cognitive behavioral therapy are not systematically falling: A revision of Johnsen and Friborg (2015).

Abstract

In a meta-analysis, Johnsen and Friborg (2015) reported a significant negative relationship between publication year and the effect sizes (ESs) of cognitive-behavioral therapy (CBT) for depressive disorders, suggesting its effectiveness was falling. We identified a series of methodological and conceptual caveats and consequently redid the meta-analysis. We used the same inclusion criteria, but only included randomized controlled trials and searched for additional eligible trials. We computed both within-group and between-group ESs for the CBT arm for the Beck Depression Inventory (BDI) and the Hamilton Rating Scale for Depression (HRSD). We assessed risk of bias, sample size, type of control group, and the study's country of origin and conducted subgroup, single, and multiple meta-regression analyses including publication year and other moderators. We identified 30 additional eligible trials. Within-group ESs presented huge heterogeneity estimates (I 2 around 90%). Year of publication was significant in some single meta-regression analyses on the BDI, but not significant in others, in most analyses on the HRSD, and in any of the analyses on between-group ESs. Multiple regression models indicated that either year was not significantly related or that both year and country were significantly related to outcomes, with a temporal trend present solely in US studies. Year of publication does not appear to be a reliable and independent moderator of the effectiveness of CBT for depression. The linear "fall" reported by Johnsen and Friborg (2015) is most likely a spurious finding. Keywords: cognitive behavioral therapy, depression, year of publication, meta-analysis, moderatorSupplemental materials: http://dx.doi.org/10.1037/bul0000062.supp A recent meta-analysis (Johnsen & Friborg, 2015) sent ripples through the psychotherapy community, when its results suggested that the effects of cognitive-behavioral therapy (CBT) for depression seem to "have declined linearly and steadily since its introduction" (p. 747). The authors conducted a meta-analysis of trials of CBT for depression and used metaregression procedures to examine the relationship between the year of publication and the effect sizes (ESs) in the CBT intervention arm, examining outcomes on two widely used depression scales: the Beck Depression Inventory (BDI) and the Hamilton Rating Scale for Depression (HRSD). Their results showed a significant decrease of the ESs over time for both the

The effect of cognitive–behavioral therapy as an antidepressive treatment is falling: Reply to Ljòtsson et al. (2017) and Cristea et al. (2017).

“…Although ITT analyses do retain all patients and thus reduce systematic attrition, they can be biased (Lane, 2008) due to undue assumptions of no change among patients that drop out. Cristea et al (2016) further argue that the ITT procedure may be less susceptible to time trend effects than completer data, but again provide no justification or evidence for this point. In contrast, our choice of using data from completers was well-informed because this was the only information available in early CBT trials.…”

“…The increased focus on historical trends in the effects of psychotherapy is likely to contribute positively to the development and implementation of more efficacious psychotherapies in the future. Some steps towards this end have been taken with the recent publication of two re-reviews of our original findings: the first being a statistical re-analysis (Ljótsson, Hedman, Mattsson, & Andersson, 2016) and the second being a meta-analytic extension (Cristea et al, 2016). We do however note considerable methodological or statistical issues in both papers, which has prompted the current reply.…”

“…Cristea et al (2016) also found some inconsistencies in Johnsen and Friborg's (2015) selection of papers-they pointed out that four of the papers should not have been included and questionable calculation of the effect sizes (ESs) for two of the included papers; however, these revisions did not change the original findings. Cristea et al (2016) introduce a number of methodological changes to the metaanalysis. For instance, they link the combination of non-RCT and RCT studies to the high degree of heterogeneity noted between studies, which we also believe may be the case.…”

“…The use of difference scores also corrects between-group ESs for any pre-treatment differences that may occur despite randomization in small sample studies. The meta-analysis of all studies combined was thus more correct in our original approach, whereas Cristea et al (2016) mix the use of post-and difference scores when analysing all studies together. A final argument for including all available studies is to ensure a substantially larger study pool, which is important for avoiding an underpowered statistical analysis and enabling weaker, yet still clinically important, statistical effects stretching across decades to appear.…”

“…The study by Cristea et al (2016) offers a comprehensive extension of the original meta-analysis as they identified a number of additional CBT studies.…”