Nearly 380,000 U.S. service members between 2000 and 2017 were, and at least 300,000 athletes annually are, diagnosed with concussion. It is imperative to establish a gold-standard diagnostic test to quickly and accurately diagnose concussion. In this non-randomized, prospective study, we examined the reliability and validity of a novel neurocognitive assessment tool, the Defense Automated Neurobehavioral Assessment (DANA), designed to be a more sensitive, yet efficient, measure of concussion symptomatology. In this study, the DANA Brief version was compared to an established measure of concussion screening, the Military Acute Concussion Evaluation (MACE), in a group of non-concussed service members. DANA Brief subtests demonstrated low to moderate reliability, as measured by intra-class correlation coefficient (ICC; values range: 0.28–0.58), which is comparable to other computerized neurocognitive tests that are widely-implemented to diagnose concussion. Statistically significant associations were found between learning and memory components of the DANA Brief and the diagnostic MACE cognitive test score (DANA Brief subtests: CDD:
R
2
= 0.05,
p
= 0.023; CDS:
R
2
= 0.10,
p
= 0.010). However, a more robust relationship was found between DANA Brief components involving attention and working memory, including immediate memory, and the MACE cognitive test score (DANA Brief subtests: GNG:
R
2
= 0.08,
p
= 0.003; PRO:
R
2
= 0.08,
p
= 0.002). These results provide evidence that the DANA Rapid version, a 5-min assessment self-administered on a hand-held portable device, based on the DANA Brief version, may serve as a clinically useful and improved neurocognitive concussion screen to minimize the time between injury and diagnosis in settings where professional medical evaluation may be unavailable or delayed. The DANA's portability, durability, shorter test time and lack of need for a medical professional to diagnose concussion overcome these critical limitations of the MACE.