Validation of a Computerized test of Functional Capacity

Abstract

Regulatory guidance for schizophrenia cognition clinical trials requires that the assessment of cognitive change is accompanied by a functionally meaningful endpoint. However, currently available measures are challenged by resistance to change, psychometric weaknesses, and for interview-based assessments, dependence upon the presence of an informant. The aims of the current study were to: 1) assess the validity, sensitivity, and reliability of the Virtual Reality Functional Capacity Assessment Tool (VRFCAT) as a measure of functional capacity; 2) determine the association between performance on the VRFCAT and performance on the MATRICS Consensus Cognitive Battery (MCCB); and 3) compare the metrics of the VRFCAT with the UCSD Performance-based Skills Assessment (UPSA). 167 patients with schizophrenia and 166 healthy controls completed the VRFCAT, UPSA, and the MCCB at baseline. The VRFCAT and UPSA were completed again at follow-up. The VRFCAT, MCCB, and UPSA were very sensitive to impairment in schizophrenia (d = 1.16 to 1.22). High test-retest reliability was demonstrated for VRFCAT total completion time and the UPSA total score in patients (ICC=0.81 and 0.78, respectively). The UPSA demonstrated significant practice effects in patients (d=0.35), while the VRFCAT did not (d=−0.04). VRFCAT total completion time was correlated with both UPSA (r=−0.56, p<0.0001 for patients and −0.58, p<0.0001 for controls) and MCCB Composite (r=−0.57, p<0.0001 for patients and −0.68, p<0.0001 for controls). The VRFCAT is a highly reliable and sensitive measure of functional capacity with associations to the UPSA and MCCB. These results provide encouraging support for a computerized functional capacity assessment for use in schizophrenia.

Assessing functional performance using computer-based simulations of everyday activities

“…The measures were also moderately correlated with standardized measures of cognition, which confirms other findings (e.g., Czaja & Sharit, 2003) that cognitive skills are important to the performance of everyday living tasks, including in patients with schizophrenia (Keefe et al, 2016). However, the fact that the correlations were moderate indicates that the measures capture other aspects of performance that do not fully overlap with standardized measures of cognition.…”

“…Currently, common FC assessment protocols used with schizophrenic patients include informant-based reports of functional performance from a relative or friend or patient themselves (Keefe et al, 2006; Ventura et al, 2015), paper and pencil based assessments such as the UPSA (Patterson et al, 2001) and more recently computer-based performance assessments such as the Virtual Reality Functional Capacity Assessment Tool (VRFCAT; Keefe et al, 2016). Although these protocols have proven useful in terms of providing information about FC they also have some limitations.…”

Virtual reality assessment of functional capacity in the early course of schizophrenia: Associations with cognitive performance and daily functioning

“…These types of sex ratio imbalances are common in studies of schizophrenia patients that use controls, given the greater incidence rate of male participants in schizophrenia studies. We did not find any evidence indicating that the sex of the participants played a significant role in their performance on the VRFCAT, nor did Keefe et al, 2016 MATRICS Battery (BACS Symbol Coding), and the Virtual Reality Functional Capacity Assessment Tool (VRFCAT). He is also a shareholder in NeuroCog Trials, Inc. and Sengenix.…”

Identifying attributes of care that may improve cost‐effectiveness in the youth mental health service system

“…148 However, there is some evidence relating to the potential of computer-based functional assessment. 149 Evidence from physical health care suggests that new collaborative technologies to promote integrated care between autonomous and geographically dispersed primary care services may help improve outcomes for patients with chronic conditions. 150…”