Creating and Validating Rapid Assessment Instruments for Practice and Research: Part 1

Abstract

In this age of accountability, social work practitioners face greater demands to demonstrate the effectiveness of their interventions. A practical way to accomplish this is to use rapid assessment instruments (RAIs) to measure a client’s level of functioning so that any subsequent change in functioning can be accurately detected over time. Although great strides have been made during the past three decades in the development of RAIs, there remains a paucity of reliable and valid scales when compared to the increasingly complex range of problems that practitioners confront in their day-to-day work with clients. This article, the first in a two-part series, outlines how to conceptualize and create a scale and how to plan and subsequently implement a design testing the newly created scale’s reliability and validity. The second article in this series provides illustrations of these steps from the authors’ own works, detailing decision making and strategies for analyzing psychometric data.

How do Runaway Adolescents and their Parents Perceive the Family? Measurement Invariance in the Family Functioning Scale

“…As this sample included paired groups, reliability was calculated separately for youths and parents [19]. Results indicated reliabilities ranged from α=0.70-0.77 for parents and from α=0.74-0.89 for adolescents across the five subscales.…”

Development and validation of a scale to measure health-related socioeconomic well-being in persons with a cancer diagnosis.

“…Carmines and Zeller (1979) state that reliabilities should not be below .80 for widely used scales, while others state that a value greater than .70 is a good indicator that the scale is measuring one attribute or concept and is helpful evaluation in development of a new scale (Grant et aI., 1990;Jacobson, 1988). Springer, Abell, and Hudson (2002) propose an even higher value of 0.80 for instruments used to make decisions about a single individual. The proposed measure of Socioeconomic…”

“…Alpha reliabilities of .80 and greater and smaller SEM are indicative of a reliable measurement tool (Faul & Van Zyl, 2004;Springer et al, 2002).…”

“…For most validation studies it is desirable to have sample sizes greater than or equal to 10 participants per item in the original pool (Springer, Abell, & Hudson, 2002). The proposed item pool for the validation study included 23 items.…”

“…Psychometric analysis of the proposed socioeconomic well-being scale was based on classical measurement theory (Nunnally & Bernstein, 1994) and followed a step-wise, iterative procedure (Faul & Van Zyl, 2004;Nunnally & Bernstein, 1994;Springer, Abell, & Hudson, 2002) Analysis began with investigating the reliability of the three domains within the proposed scale (material, human, and social capitol), via the alpha coefficient and standard error of measurement (SEM). Alpha reliabilities of .80 and greater and smaller SEM are indicative of a reliable measurement tool (Faul & Van Zyl, 2004;Springer et al, 2002).…”