Evidence for Added Value of Baseline Testing in Computer-Based Cognitive Assessment

Abstract

Context: Large-scale baseline cognitive assessment for individuals at risk for concussion is a common part of the protocol for concussion-surveillance programs, particularly in sports. Baseline cognitive testing is also being conducted in US military service members before deployment. Recently, the incremental validity of large-scale baseline cognitive assessment has been questioned.Objective: To examine the added value of baseline cognitive testing in computer-based neuropsychological assessment by comparing 2 methods of classifying atypical performance in a presumed healthy sample.Design: Cross-sectional study. Main Outcome Measure(s): Rates of atypical performance in this healthy, active-duty sample were determined first by comparing postdeployment scores with a military normative database and then with each individual's personal baseline performance using a reliable change index.Results: Overall rates of atypical performance were comparable across these 2 methods. However, these methods were highly discordant in terms of which individuals were classified as atypical. When norm-referenced methods were used, 2.6% of individuals classified as normal actually demonstrated declines from baseline. Further, 65.7% of individuals classified as atypical using norm-referenced scores showed no change from baseline (ie, potential false-positive findings).Conclusions: Knowing an individual's baseline performance is important for minimizing potential false-positive errors and reducing the risks and stresses of misdiagnosis.Key Words: Automated Neuropsychological Assessment Metrics, computerized neuropsychological testing, concussions, mild traumatic brain injuries, military athletes Key PointsWhen service members' predeployment and postdeployment performances on the Automated Neuropsychological Assessment Metrics test were compared using norm-referenced scores and their personal baselines, the absolute rates of atypical performance were similar. However, for those individuals whose performance was classified as atypical, a high degree of discordance was noted between the methods. Using norms alone resulted in a high level of false-positive errors. Mistakenly classifying an individual as cognitively impaired should be avoided whenever possible because it can lead to overuse of medical resources and undue emotional stress. C omputer-based cognitive testing is fast becoming standard practice in the assessment and management of mild traumatic brain injury (mTBI) in sports and military concussion-management programs. Recommended sport-concussion protocols typically include baseline assessment of cognitive performance for individuals at high risk for concussion, against which postconcussion performance is compared.

Normative Versus Baseline Paradigms for Detecting Neuropsychological Impairment Following Sports-Related Concussion

“…A central limitation to the present study also affects all previous studies comparing baseline to normative paradigms (Echemendia et al, 2012a; Louey et al, 2014; Roebuck-Spencer et al, 2013; Schmidt et al, 2012). The present and prior studies have all utilised a repeated-measures design, with concussed and control athletes completing baseline and post-concussion assessments.…”

“…Yet, when considering classification of individual athletes the baseline paradigm was more sensitive than the normative paradigm, with comparable specificity levels. From the non-sporting literature, Roebuck-Spencer, Vincent, Schlegel, and Gilliland (2013) compared the baseline and normative paradigms in a large sample ( n = 8002) of non-injured military personnel on the ANAM. Whilst they found that the overall false-positive rate was comparable for both paradigms, they demonstrated that 65.7% of individuals classified as atypical using the normative paradigm showed no change under the baseline paradigm.…”

History of neuropsychological study of sport-related concussion

“…Very recently, discussion has arisen in the professional literature regarding whether baseline testing sufficiently improves the accuracy of clinical decision-making in the management of sport concussions so as to justify the extra effort involved [15][16][17][18][19]. In part, this discussion was spurred by Randolph et al's [20] comments in 2005, in which they concluded that neuropsychological assessment generally (including baseline testing) had not been shown empirically to add value in concussion management.…”

“…In contrast to these two studies are two reports that greatly favoured baseline vs. normative comparison. Using a sample of 8000 military personnel, Roebuck-Spencer et al [18] compared detection of post-deployment impairment on the ANAM for baseline vs. normative comparison methods. The authors demonstrated that the baseline comparison method more accurately identified post-deployment cognitive deficits than did comparison to norms and also classified as normal (compared to baseline) 68% of individuals whom the normative comparison classified as atypical.…”

Diagnosis and Management of Acute Concussion

“…Furthermore, they saw that a large number of those classified as atypical were no different in postdeployment from their baseline, suggesting that using normative standards falsely identified these individuals as atypical. 135 However, due to ease of administration, some athletic programs are solely utilizing computerized neurocognitive tests for making decisions regarding return to activity. All current evidence-and consensus-based sports concussion recommendations advise against having a single test to diagnose or manage concussion, and that these tests should be used in conjunction with other evaluation modalities to make diagnostic and management decisions.…”