Regulatory guidance for schizophrenia cognition clinical trials requires that the assessment of cognitive change is accompanied by a functionally meaningful endpoint. However, currently available measures are challenged by resistance to change, psychometric weaknesses, and for interview-based assessments, dependence upon the presence of an informant. The aims of the current study were to: 1) assess the validity, sensitivity, and reliability of the Virtual Reality Functional Capacity Assessment Tool (VRFCAT) as a measure of functional capacity; 2) determine the association between performance on the VRFCAT and performance on the MATRICS Consensus Cognitive Battery (MCCB); and 3) compare the metrics of the VRFCAT with the UCSD Performance-based Skills Assessment (UPSA). 167 patients with schizophrenia and 166 healthy controls completed the VRFCAT, UPSA, and the MCCB at baseline. The VRFCAT and UPSA were completed again at follow-up. The VRFCAT, MCCB, and UPSA were very sensitive to impairment in schizophrenia (d = 1.16 to 1.22). High test-retest reliability was demonstrated for VRFCAT total completion time and the UPSA total score in patients (ICC=0.81 and 0.78, respectively). The UPSA demonstrated significant practice effects in patients (d=0.35), while the VRFCAT did not (d=−0.04). VRFCAT total completion time was correlated with both UPSA (r=−0.56, p<0.0001 for patients and −0.58, p<0.0001 for controls) and MCCB Composite (r=−0.57, p<0.0001 for patients and −0.68, p<0.0001 for controls). The VRFCAT is a highly reliable and sensitive measure of functional capacity with associations to the UPSA and MCCB. These results provide encouraging support for a computerized functional capacity assessment for use in schizophrenia.

Computerized tests benefit from automated scoring procedures and standardized administration instructions. These methods can reduce the potential for rater error. However, especially in patients with severe mental illnesses, the equivalency of traditional and tablet-based tests cannot be assumed. The Brief Assessment of Cognition in Schizophrenia (BACS) is a pen-and-paper cognitive assessment tool that has been used in hundreds of research studies and clinical trials, and has normative data available for generating age- and gender-corrected standardized scores. A tablet-based version of the BACS called the BAC App has been developed. This study compared performance on the BACS and the BAC App in patients with schizophrenia and healthy controls. Test equivalency was assessed, and the applicability of paper-based normative data was evaluated. Results demonstrated the distributions of standardized composite scores for the tablet-based BAC App and the pen-and-paper BACS were indistinguishable, and the between-methods mean differences were not statistically significant. The discrimination between patients and controls was similarly robust. The between-methods correlations for individual measures in patients were r>0.70 for most subtests. When data from the Token Motor Test was omitted, the between-methods correlation of composite scores was r=0.88 (df=48; p<0.001) in healthy controls and r=0.89 (df=46; p<0.001) in patients, consistent with the test-retest reliability of each measure. Taken together, results indicate that the tablet-based BAC App generates results consistent with the traditional pen-and-paper BACS, and support the notion that the BAC App is appropriate for use in clinical trials and clinical practice.

Background: Continuing advances in the understanding of Alzheimer’s disease progression have inspired development of disease-modifying therapeutics intended for use in preclinical populations. However, identification of clinically meaningful cognitive and functional outcomes for individuals who are, by definition, asymptomatic remains a significant challenge. Clinical trials for prevention and early intervention require measures with increased sensitivity to subtle deficits in instrumental activities of daily living (IADL) that comprise the first functional declines in prodromal disease. Validation of potential endpoints is required to ensure measure sensitivity and reliability in the populations of interest. Objectives: The present research validates use of the Virtual Reality Functional Capacity Assessment Tool (VRFCAT) for performance-based assessment of IADL functioning in older adults (age 55+) with subjective cognitive decline. Design: Cross-sectional validation study. Setting: All participants were evaluated on-site at NeuroCog Trials, Durham, NC, USA. Participants: Participants included 245 healthy younger adults ages 20-54 (131 female), 247 healthy older adults ages 55-91 (151 female) and 61 older adults with subjective cognitive decline (SCD) ages 56-97 (45 female). Measures: Virtual Reality Functional Capacity Assessment Tool; Brief Assessment of Cognition App; Alzheimer’s Disease Cooperative Study Prevention Instrument Project – Mail-In Cognitive Function Screening Instrument; Alzheimer’s Disease Cooperative Study Instrumental Activities of Daily Living – Prevention Instrument, University of California, San Diego Performance-Based Skills Assessment – Validation of Intermediate Measures; Montreal Cognitive Assessment; Trail Making Test- Part B. Results: Participants with SCD performed significantly worse than age-matched normative controls on all VRFCAT endpoints, including total completion time, errors and forced progressions (p≤0001 for all, after Bonferonni correction). Consistent with prior findings, both groups performed significantly worse than healthy younger adults (age 20-54). Participants with SCD also performed significantly worse than controls on objective cognitive measures. VRFCAT performance was strongly correlated with cognitive performance. In the SCD group, VRFCAT performance was strongly correlated with cognitive performance across nearly all tests with significant correlation coefficients ranging from 0.3 to 0.7; VRFCAT summary measures all had correlations greater than r=0.5 with MoCA performance and BAC App Verbal Memory (p<0.01 for all). Conclusions: Findings suggest the VRFCAT provides a sensitive tool for evaluation of IADL functioning in individuals with subjective cognitive decline. Strong correlations with cognition across groups suggest the VRFCAT may be uniquely suited for clinical trials in preclinical AD, as well as longitudinal investigations of the relationship between cognition and function.

No abstract

Despite evidence that it exists, short-term within-individual variability in cognitive performance has largely been ignored as a meaningful component of human cognitive ability. In this article, we build a case for why this within-individual variability should not be viewed as mere measurement error and why it should be construed as a meaningful component of an individual’s cognitive abilities. We argue that in a demanding and rapidly changing modern world, between-individual analysis of single-occasion cognitive test scores does not account for the full range of within-individual cognitive performance variation that is implicated in successful typical cognitive performance. We propose that short-term repeated-measures paradigms (e.g., the experience sampling method (ESM)) be used to develop a process account of why individuals with similar cognitive ability scores differ in their actual performance in typical environments. Finally, we outline considerations for researchers when adapting this paradigm for cognitive assessment and present some initial findings from two studies in our lab that piloted the use of ESM to assess within-individual cognitive performance variation.