Accuracy of a Rationally Derived Method for Identifying Treatment Failure in Children and Adolescents

Abstract

Psychotherapy outcome can be enhanced by early identification of potential treatment failures before they leave treatment. In adults, compelling data are emerging that provide evidence that an early warning system that identifies potential treatment failures can be developed and applied to enhance outcome. The present study reports an analysis of early warning algorithms to identify treatment failures among child/adolescent patients (ages 3-18). The progress of 300 patients who had completed treatment was analyzed to see if algorithms could identify those children who ultimately had a negative outcome. Results indicated that the rationally derived method had a 77% success rate for identifying child/adolescent patients who were reliably worse or had deteriorated by the time that therapy was terminated.

Identifying Youth at Risk for Treatment Failure in Outpatient Community Mental Health Services

“…Psychotherapy research suggests that 5-10% of adult psychotherapy clients can be classified as experiencing deterioration or treatment failure-leaving treatment significantly worse off than when they entered (Lambert and Bergin 1994;Mohr 1995). Similar estimates of deterioration have been found for child and adolescent populations in managed care settings (Bishop et al 2005;Bybee et al 2007), and rates may be even higher for children and adolescents in traditional community mental health settings (Weisz et al 1995). In a related vein, Lilienfeld (2007) asserted that greater emphasis should be placed on research identifying potentially harmful treatments than on identifying empiricallysupported therapies.…”

“…Two studies with child and adolescent samples suggest that promising results may also be expected with younger patients. Bishop et al (2005) tested the accuracy of rationallyderived algorithms-those based on expert opinion and outcome measure characteristics-for identifying potential treatment failures in a sample of 300 residential and outpatient clients ages 3-18. Overall, this rationally-derived method was successful in identifying 77% of child/adolescent patients who had deteriorated by the end of treatment.…”

Predicting patient deterioration in youth mental health services: community mental health vs. managed care settings

“…For example, Lambert, Whipple, Bishop et al (2002) reported overall hit rates with adult patients of .79 and .83 for rationally derived and empirically derived methods, respectively. Similarly, Bishop et al (2005) reported an overall hit rate of .81 for a combined sample of youth served in residential and outpatient settings, and Bybee et al (2007) achieved an overall hit rate of .88 for youth in outpatient managed care settings. Relative to these studies, the overall hit rate for the present study (.75) may be a conservative estimate due to the methodology we employed for including scores that could be used as an alert signal.…”

“…A limited number of studies have tested the accuracy of rationally derived or empirically derived methods for identifying youth at risk for treatment failure, each using versions of the Youth Outcome Questionnaire (Y-OQ; Burlingame et al, 2001;Burlingame, Wells, Lambert, & Cox, 2004;Burlingame et al, 2005). For example, Bishop et al (2005) examined the accuracy of a rationally derived method (i.e., warning system algorithms based on expert opinion and characteristics of the outcome measure) for identifying youth at risk for treatment failure in a managed care system. In a sample of 145 residential patients and 155 outpatient youth, the authors found accuracy rates comparable to those found in adult studies for identifying deteriorating cases (77%); however, accuracy of identifying deteriorators was notably lower for the outpatient group (72.3%) than for the residential sample (81.8%).…”

Curves of expected recovery and their predictive validity for identifying treatment failure

“…Psychotherapy treatment for child and adolescent populations is generally equal to adults in terms of effect sizes reported (Durlak & McGlinchey, 1999;Weisz, Weiss & Donenberg, 2002). To date there is sparse research on deterioration and on validating treatment response feedback technologies for child and adolescent populations (Bishop, Bybee, Lambert, Burlingame & Wells, 2005). Yet, the need for ongoing continuous assessment in the treatment of child and adolescent populations has been issued (Kazdin, 2001).…”