A preliminary evaluation of the reliability and validity of a self‐reported communicative functioning item pool

Abstract

Background: Item banking, an approach to test development based in item response theory (IRT), is beginning to be applied to the measurement of communicative functioning in aphasia. This approach involves calibrating a set of test items responding to a particular latent trait to a common measurement scale. One method for validating such scales is to examine the degree to which obtained item calibration estimates agree with a priori item rankings based on theory or expert opinion. Aims: The purpose of this study was to evaluate the utility of magnitude estimation (ME) procedures for validating item banks containing self-reported functional performance items, and to make a preliminary analysis of the validity of a proposed item pool for measuring self-reported communicative functioning in aphasia. Methods & Procedures:A total of 14 raters made ME ratings of physical and communicative functioning items. These ratings were evaluated for their intra and interobserver reliability and, for subsets of the items, their correspondence with previously published IRT calibration estimates was also evaluated. Outcomes & Results: Intra-rater reliability was moderate to high, and inter-rater reliability was high. Correspondence with IRT calibrations was high for physical items, and moderate for communication items. The distribution of ME ratings for the communication items was negatively skewed. Conclusions: ME procedures have utility for investigating the validity of functional performance items. The results suggest that communicative functioning may have a more complex latent structure than physical functioning, and that the proposed item pool might benefit from the inclusion of additional items at the lower end of the scale.

Self-perceptions of functional communication performance during total communication intervention

“…Because such an index gives only a general picture over the functional communication performance it was important to look at it in more detail. According to Hula, Austermann Hula, and Doyle (2009) the construct of communicative functioning, a latent trait that we are trying to define, can have a complex structure. Lomas et al (1989) describe how the functional communication performance measured with CETI comprises communicating basic needs, social needs, life skills, and health threat.…”