A comparison of rational versus empirical methods in the prediction of psychotherapy outcome

Abstract

Recent studies have shown that providing feedback to therapists based on comparing their clients' progress to a set of rational, clinically derived algorithms delineating various categories of progress has enhanced outcomes for clients predicted to show poor treatment outcomes (signal-alarms). One prior study indicated that an empirically derived method more accurately predicts outcome than the rational method. The present study replicated the comparison of empirical and rational methods, while adding an additional layer of effect size analyses to further clarify predictive accuracy. The two methods were approximately equivalent in their accurate detection of cases that had a final negative outcome. However, the rational method had significantly lower overall predictive accuracy due to its high percentage of false negative predictions. Further, progressively more optimistic predictions based on the empirical method were associated with greater improvement for the average client. This was not the case for the rational method.

Maximizing Psychotherapy Outcome beyond Evidence-Based Medicine

“…Finch et al [13] developed and tested the accuracy of predicting deterioration in adults. These and related signal alarm systems have been evaluated in a number of empirical investigations [14,15,16,17]. This research shows that 85-100% of patients who deteriorate in treatment can be accurately identified prior to departing from treatment and often after the first few sessions of treatment.…”

Predicting patient deterioration in youth mental health services: community mental health vs. managed care settings

“…Furthermore, these estimates should be viewed with caution given the low number of actual deteriorators on which they were based (11 in each sample), and because the selection criteria for inclusion in the study analyses yielded a sample of only 6.6% of the original patient database. In addition, comparable research in adults suggests that empirically derived algorithms for predicting treatment failure (those using empirical outcome data and average change trajectories for patients with similar symptom levels) tend to be more accurate than those that use rationally derived methods (Lambert, Whipple, Bishop et al, 2002;Spielmans, Masters, & Lambert, 2006). Subsequently, Bybee, Lambert, and Eggett (2007) tested an empirically derived method (based on multilevel modeling of empirically based change trajectories) for identifying youth at risk for treatment failure in an outpatient managed care setting, accurately identifying 72% of deteriorators.…”

Improving psychotherapy outcome: the use of immediate electronic feedback and revised clinical support tools

“…A client becomes a NOT case by virtue of being so far from the trajectory of expected treatment response as to be identifi ed by algorithms that have been shown to accurately identify 85-100% of cases that deteriorate (Finch et al, 2001;Lambert, Whipple, Bishop et al, 2002;Speilman et al, 2006). In past studies, progress feedback to therapists of OT clients has not proved to be benefi cial; therefore, no hypotheses about the outcome for these clients are proposed.…”

“…Lambert and colleagues continue an ongoing research programme aimed at reducing treatment failure by providing therapists and clients with weekly progress feedback, which includes alerts based on statistically generated algorithms to identify cases at risk for deterioration (Finch, Lambert, & Schaalje, 2001;Lambert, Whipple, Bishop et al, 2002;Speilman, Masters, & Lambert, 2006). Using the algorithms, clients who are progressing as expected are designated as 'on track' (OT) and clients who are not are designated as 'not on track' (NOT), thus allowing therapists to focus attention on these failing cases and to enhance their likelihood of success.…”