AKL-T01 is a digital therapeutic (DTx) that targets attention by generating conflict at dynamically updated difficulty levels during a multitasking game. Clinical trials support AKL-T01’s efficacy in attention-deficit/hyperactivity disorder (ADHD), but there is a need to understand how in-game data can be used to monitor patient changes in cognition. We aimed to derive a real-time measure of attention from AKL-T01 gameplay data and validate it against clinical outcomes. Trials of AKL-T01 included: STARS-ADHD-Adult (NCT05183919), a 6-week trial in adults 18 and older (
n
= 221;
M
age = 39.9; 70% female); STARS-ADHD-Adolescent (NCT04897074), a 4-week trial in adolescents ages 13–17 (
n
= 162;
M
age = 14.4; 41% female); and STARS-ADHD (NCT02674633), a 4-week trial in children ages 8–12 (
n
= 180;
M
age = 9.7; 31% female). A cognitive metric was derived from targeting response speed, targeting sensitivity (d-prime), and navigation skill level. Using multiple linear regression models, we analyzed relationships between cognitive metric change and changes in the Test of Variables of Attention (TOVA)-Attention Comparison Score (ACS), controlling for TOVA-ACS baseline, cognitive metric baseline, age, and sex. We explored associations with ADHD symptoms and quality of life. Increases in the cognitive metric significantly predicted increases in TOVA-ACS in the adult (
β
= 0.16,
p
< 0.001), adolescent (
β
= .09,
p
= 0.007), and pediatric (
β
= 0.06,
p
= 0.014) trials. Cognitive metric changes additionally related to self-reported quality of life in adults and clinician-rated ADHD symptoms in adolescents. Findings support the clinical validity of a real-time measure of attention derived from AKL-T01 patient-device interactions.