Test–Retest Reliability of Concussion Baseline Assessments in United States Service Academy Cadets: A Report from the National Collegiate Athletic Association (NCAA)–Department of Defense (DoD) CARE Consortium

Abstract

Objective: In response to advancing clinical practice guidelines regarding concussion management, service members, like athletes, complete a baseline assessment prior to participating in high-risk activities. While several studies have established test stability in athletes, no investigation to date has examined the stability of baseline assessment scores in military cadets. The objective of this study was to assess the test–retest reliability of a baseline concussion test battery in cadets at U.S. Service Academies. Methods: All cadets participating in the Concussion Assessment, Research, and Education (CARE) Consortium investigation completed a standard baseline battery that included memory, balance, symptom, and neurocognitive assessments. Annual baseline testing was completed during the first 3 years of the study. A two-way mixed-model analysis of variance (intraclass correlation coefficent (ICC)3,1) and Kappa statistics were used to assess the stability of the metrics at 1-year and 2-year time intervals. Results: ICC values for the 1-year test interval ranged from 0.28 to 0.67 and from 0.15 to 0.57 for the 2-year interval. Kappa values ranged from 0.16 to 0.21 for the 1-year interval and from 0.29 to 0.31 for the 2-year test interval. Across all measures, the observed effects were small, ranging from 0.01 to 0.44. Conclusions: This investigation noted less than optimal reliability for the most common concussion baseline assessments. While none of the assessments met or exceeded the accepted clinical threshold, the effect sizes were relatively small suggesting an overlap in performance from year-to-year. As such, baseline assessments beyond the initial evaluation in cadets are not essential but could aid concussion diagnosis.

Measuring test-retest reliability (TRR) of AMSTAR provides moderate to perfect agreement – a contribution to the discussion of the importance of TRR in relation to the psychometric properties of assessment tools

“…A shorter timespan might have provided different results. Other studies investigating TRR used various ranges of timespans, such as weeks, months and even years [24,25,28,29]. In our study, all reviewers were experienced in using AMSTAR, and the learning curve might explain only a small degree of the variation.…”

“…On the other hand, if the timespan is too long, the ratings might be affected by the learning curve of the reviewers as well by changes in the field, such as the development of new assessment instruments [22,23]. Studies in other research fields used varying timespans when measuring TRR, such as two weeks, 18 months and two years [24,25]. The optimal timespan depends on the context in which the testing takes place.…”

Proteomic Discovery of Noninvasive Biomarkers Associated With Sport-Related Concussions

“…To date, most commonly used diagnostic assessments rely on a combination of subjective measures such as patient-reported symptoms, balance, or neurocognitive function, which have been shown to lack significant clinical reliability. 4,47 Therefore, the need for an objective diagnostic is critical. Here, we compared healthy control collegiate athletes to separate collegiate athletes who had sustained a concussion.…”

Leadership Lessons in Concussion Management for Team Physicians

“…Lastly, the Team Physician should make a decision regarding how often (ie, annually, biennial, upon entry to the organization) the preparticipation assessments are to be collected. With substantial overlap observed between annual assessments in college athletes32 and military cadets,33 baseline collections beyond the initial entry evaluation are not essential. However, same season baseline assessments have aided concussion diagnoses 30.…”