The Effects of Repeated Administrations on Self-Report and Parent-Report Scales

Abstract

The use of repeated measures to track the out-come of psychosocial interventions is becoming common practice in most organizations. One threat with this methodology is the retest artifact. This artifact is defined by a systematic change in scores for individuals who repeatedly complete an outcome instrument yet do not receive an) formal treatment. The retest artifact can call into question the validity of change associated with school-based treatment programs when this change is based upon repeated testing using outcome instruments. This study directly assessed the retest effect for both a youth and an adult psychosocial outcome measure. The frequency of retest administrations was systematically varied (weekly, biweekly, monthly, pre/post testing) over a 9-week period using a youth parent-report and an adult self-report Outcome Questionnaire. Results indicate a statistically significant improvement in functioning at the time of the second administration regardless of the lapse in time for the adult measure and a steady improvement in functioning for the parent-report youth measure. Implications of this finding for evaluating psychosocial interventions are discussed.

Accuracy of a Rationally Derived Method for Identifying Treatment Failure in Children and Adolescents

“…The YOQ was used for both signal-alarm alerts and for classification of final status. Some research (e.g., Durham et al, 2002) suggests that a test retest artifact with improving YOQ scores over administration exists, but also that this artifact is much smaller than the Reliable Change Index that was used as a marker of clinically significant change in the present study. Further research is needed to examine the accuracy of predicting outcome based on comprehensive measures of outcome drawn from a variety of reporting sources such as schoolbased performance, teacher ratings, peer ratings, societal records, and the like.…”

The CORE‐10: A short measure of psychological distress for routine use in the psychological therapies

“…Regardless of the measure, there needs to be further research into the impact on clients as well as the psychometrics of such repeated use (e.g. Durham et al, ). For example, Longwell and Truax () found that the weekly administration of the Beck Depression Inventory‐II over a two‐month period in a nonclinical population resulted in significantly lower estimates of depression.…”

Improving the effects of psychotherapy: The use of early identification of treatment and problem-solving strategies in routine practice.

“…Furthermore, the OQ-45 has been shown to be sensitive to change in clients over short time periods while remaining stable in untreated individuals (Vermeersch, Lambert, & Burlingame, 2000). Repeated testing of persons has been found to have little effect on OQ-45 test scores (Durham et al, 2002).…”