Benchmarking the Effectiveness of Psychotherapy: Program Evaluation as a Component of Evidence-Based Practice

Abstract

The medical director of a child guidance center is starting a new treatment program. Following intense media coverage of adverse events associated with selective serotonin reuptake inhibitors, depressed teens and their families are refusing to accept pharmacotherapy. The director has paid for three social work therapists to attend a cognitivebehavioral therapy (CBT) workshop, and these clinicians will begin seeing depressed teens next week. The director, however, is worried. From what she has heard and read, CBT has a good track record in treating depression although it may work best in combination with the medications that her patients are refusing to take. In addition, she wonders how well will CBT work when delivered by her social work therapists to the population of poor, Spanishspeaking teens served by the clinic?In this example, the medical director struggles with how to bring the principles of evidence-based practice (EBP) to bear on the problem of program evaluation. She is asking a critical, pragmatic question: Does therapy work, here, now, and with my patients? This column discusses a method for addressing this query that uses the results of published psychotherapy clinical trials as a gold-standard benchmark against which the outcomes of practice can be measured (McFall, 1996). This methodology views research evidence on the effects of psychotherapy as, literally, a base-solid ground that researchers and practitioners can use as a foundation when trying to solve problems, whether theoretical or applied.

Structuring the Collaboration of Science and Service in Pursuit of a Shared Vision

“…Clinical dashboards present case context, progress, and practice history on a single display. Common messages that may be collected and represented on dashboards 336 CHORPITA AND DALEIDEN include (a) observed values from assessment events with youth, families, providers, or other parties; (b) treatment team practice plans or progress benchmarks for celebration or additional review; (c) research benchmarks of clinical cutoff scores, expected rates of change (e.g., Weersing, 2005), or expected best practice events such as session sequences from a treatment manual or practice elements retrieved from a PWEBS search; and (d) administrative indicators for change in eligibility status, time or volume-based utilization triggers for reauthorization or intensive review, and so on. Essentially, dashboards are a telecommunication tool that support feedback, feed forward, exploration, and simulation.…”

Family‐based treatment for adolescent substance abuse: controlled trials and new horizons in services research

“…Benchmarks can be derived from many sources, including local or nationwide performance standards (e.g. Weersing, 2005), national warehouse databases (Mellor-Clark et al , 2006; Mullin et al , 2006), or treatment efficacy trials in the form of single landmark studies (Gaston et al , 2006) or a group of studies aggregated via quantitative review (Chorpita et al , 2002) or meta-analyses (Minami et al , 2007). …”

The Brief Child and Family Phone Interview (BCFPI): 1. Rationale, development, and description of a computerized children’s mental health intake and outcome assessment tool

“…Children’s mental health professionals are also increasingly accountable for service outcomes (Hoagwood et al, 1996). Repeating standardized intake measures at treatment and termination permits service providers to compare the effectiveness of locally developed programs or recently adopted evidence‐based treatments to the outcome of clinical trials (Weersing, 2005). Linking standardized outcome measures to costs permits the relative cost‐effectiveness of different service options to be determined (Romeo, Byford, & Knapp, 2005).…”