The reliability and validity of qualitative scores for the Controlled Oral Word Association Test

Abstract

The reliability and validity of two qualitative scoring systems for the Controlled Oral Word Association Test [Benton, A. L., Hamsher, de S. K., & Sivan, A. B. (1983). Multilingual aplasia examination (2nd ed.). Iowa City, IA: AJA Associates] were examined in 108 healthy young adults. The scoring systems developed by Troyer et al. [Troyer, A. K., Moscovich, M., & Winocur, G. (1997). Clustering and switching as two components of verbal fluency: Evidence from younger and older healthy adults. Neuropsychology, 11, 138-146] and by Abwender et al. [Abwender, D. A., Swan, J. G., Bowerman, J. T., & Connolly, S. W. (2001a). Qualitative analysis of verbal fluency output: Review and comparison of several scoring methods. Assessment, 8, 323-336] each demonstrated excellent interrater reliability (all indices at or above r(icc)=.9). Consistent with previous research [e.g., Ross, T. P. (2003). The reliability of cluster and switch scores for the COWAT. Archives of Clinical Psychology, 18, 153-164), test-retest reliability coefficients (N=53; M interval 44.6 days) for the qualitative scores were modest to poor (r(icc)=.6 to .4 range). Correlations among COWAT scores, measures of executive functioning, verbal learning, working memory, and vocabulary were examined. The idea that qualitative scores represent distinct executive functions such as cognitive flexibility or strategy utilization was not supported. We offer the interpretation that COWAT performance may require the ability to retrieve words in a non-routine manner while suppressing habitual responses and associated processing interference, presumably due to a spread of activation across semantic or lexical networks. This interpretation, though speculative at present, implies that clustering and switching on the COWAT may not be entirely deliberate, but rather an artifact of a passive (i.e., state-dependent) process. Ideas for future research, most noticeably experimental studies using cognitive methods (e.g., priming), are discussed.

Methodological Criteria for Scoring Clustering and Switching in Verbal Fluency Tasks

“…Furthermore, these authors argued that switching between single words (hard switching) reflects only an inability to perform clustering, whereas switching between clusters (cluster switching) reflects cognitive flexibility and switching skills, thus distinguishing between these two scores. However, Ross et al (2007) compared the two methods (Troyer et al, 1997;Abwender et al, 2001) to check the reliability and validity of the qualitative analysis methods. The authors found good reliability rates related to the results of the original study by Abwender et al (2001); however, there was no evidence that this method of analysis was better than the method proposed by Troyer et al (1997).…”

“…Therefore, we chose to keep the scoring used in the original method. Nevertheless, future studies should further investigate the reliability and validity of the method because no consensus has been reached (Ross et al, 2007).…”

The FAS fluency test in Brazilian children and teenagers: executive demands and the effects of age and gender

“…We can suppose that further differentiation of EF components occurs across development 12,21,32 , along with verbal fluency becoming more dependent on such components; more elementary abilities (such as vocabulary) become more stable over time, and differences in performance across age become attributable to more complex abilities, such as EFs. Indeed, some stu dies have found a relationship between vocabulary and phonemic fluency tasks in children aged 7-9 years 32 , but not in youths 33 . Vocabulary demands should be addressed and con trolled in future research.…”

Subthalamic stimulation affects homophone meaning generation in Parkinson's disease

“…The HMGT stresses semantic switching skills but performance might also be affected by the lexical retrieval demands of providing definitions. Category fluency tasks require switching semantic subdomains within a larger semantic domain according to strict semantic criteria, phonemic fluency tasks require switching between semantic domains on the basis of lexical search (Troyer et al, 1998); and both tasks require an ability to suppress habitual responses (Perrett, 1974;Ross et al, 2007). STN stimulation has previously been reported to improve the ability to inhibit dominant semantic meanings (as measured by the Hayling Test) (Castner et al, 2007).…”