The reliability and validity of the Complex Task Performance Assessment: A performance-based assessment of executive function

Abstract

The objective of this study was to evaluate the inter-rater reliability, test-retest reliability, concurrent validity, and discriminant validity of the Complex Task Performance Assessment (CTPA): An ecologically-valid performance-based assessment of executive function. Community control participants (n = 20) and individuals with mild stroke (n = 14) participated in this study. All participants completed the CTPA and a battery of cognitive assessments at initial testing. The control participants completed the CTPA at two different times one week apart. The intra-class correlation coefficient (ICC) for inter-rater reliability for the total score on the CTPA was 0.991. The ICCs for all of the sub scores of the CTPA were also high (0.889-0.977). The CTPA total score was significantly correlated to Condition 4 of the DKEFS Color-Word Interference Test (ρ = −0.425), and the Wechsler Test of Adult Reading (ρ = −0.493). Finally, there were significant differences between control subjects and individuals with mild stroke on the total score of the CTPA (p = 0.007) and all sub scores except interpretation failures and total items incorrect. These results are also consistent with other current executive function performance-based assessments and indicate that the CTPA is a reliable and valid performance-based measure of executive function.

Development and Alternate Form Reliability of the Complex Task Performance Assessment (CTPA) for People With Mild Stroke

“…Although our sample showed significant improvement in overall score when taking the CTPA-Alt first, comparison of z scores of the overall scores indicates that the learning effect was markedly less than that reported with serial administration of the CTPA (Wolf et al, 2015). Even so, special consideration should be given to the order of administration of the CTPA versions to maximize utility and minimize learning effects; the original version of the CTPA should be given first.…”

“…Learning effects make serial administration of the CTPA undesirable (Wolf et al, 2015), so the CTPA-Alt provides an additional form of this instrument to use in detecting mild impairments in cognition. Although our sample showed significant improvement in overall score when taking the CTPA-Alt first, comparison of z scores of the overall scores indicates that the learning effect was markedly less than that reported with serial administration of the CTPA (Wolf et al, 2015).…”

“…In the CTPA, two tasks occur simultaneously: (1) calculation of library clients' fees (inventory control) and (2) telephone messaging. Specific details about the CTPA have been previously published (Wolf et al, 2015), and the CTPA is available in the public domain (see Wolf, Morrison, & Matheson, 2011).…”

“…The PHQ-9 and D-KEFS had been used for participant inclusion in previous studies with the CTPA (Wolf et al, 2008(Wolf et al, , 2015. The PHQ-9 consists of nine questions addressing common symptoms of depression; results range from no depression (1-4 points) to severe depression (20-27 points).…”

“…The validity of the CTPA was published for a sample of people with executive dysfunction after mild stroke; however, a second administration of the CTPA revealed strong learning effects (Wolf, Dahl, Auen, & Doherty, 2015), contraindicating use of the CTPA as an outcome measure to evaluate recovery or the impact of an intervention. The aims of this study were to develop an alternate form of the CTPA (CTPA-Alt) and to evaluate the alternate form reliability of the CTPA-Alt.…”