Computerized neurocognitive assessment tools (NCATs) are increasingly used for baseline and post-concussion assessments. To date, NCATs have not demonstrated strong test-retest reliabilities. Most studies have used non-military populations and different methodologies, complicating the determination of the utility of NCATs in military populations. The test-retest reliability of four NCATs (Automated Neuropsychological Assessment Metrics 4 [ANAM4], CNS-Vital Signs, CogState, and Immediate Post-Concussion Assessment and Cognitive Test [ImPACT]) was investigated in a healthy active duty military sample. Four hundred and nineteen Service Members were randomly assigned to take one NCAT and 215 returned after approximately 30 days for retest. Participants deemed to have inadequate effort during one or both testing sessions, according to the NCATs scoring algorithms, were removed from analyses. Each NCAT had at least one reliability score (intraclass correlation) in the "adequate" range (.70-.79), only ImPACT had one score considered "high" (.80-.89), and no scores met "very high" criteria (.90-.99). However, overall test-retest reliabilities in four NCATs in a military sample are consistent with reliabilities reported in the literature and are lower than desired for clinical decision-making.

Computerized neurocognitive assessment tools (NCATs) offer potential advantages over traditional neuropsychological tests in postconcussion assessments. However, their psychometric properties and clinical utility are still questionable. The body of research regarding the validity and clinical utility of NCATs suggests some support for aspects of validity (e.g., convergent validity) and some ability to distinguish between concussed individuals and controls, though there are still questions regarding the validity of these tests and their clinical utility, especially outside of the acute injury timeframe. In this paper, we provide a comprehensive summary of the existing validity literature for four commonly used and studied NCATs (automated neuropsychological assessment metrics, CNS vital signs, cogstate and immediate post-concussion and cognitive testing) and lay the groundwork for future investigations.

Though the results are not overly promising for the validity of the four NCATs we investigated, traditional methods of investigating psychometric properties may not be appropriate for computerized tests. We offer several conceptual and methodological considerations for future studies regarding the validity of NCATs.