Puzzling With Online Games (BAM-COG): Reliability, Validity, and Feasibility of an Online Self-Monitor for Cognitive Performance in Aging Adults

Abstract

BackgroundOnline interventions are aiming increasingly at cognitive outcome measures but so far no easy and fast self-monitors for cognition have been validated or proven reliable and feasible.ObjectiveThis study examines a new instrument called the Brain Aging Monitor–Cognitive Assessment Battery (BAM-COG) for its alternate forms reliability, face and content validity, and convergent and divergent validity. Also, reference values are provided.MethodsThe BAM-COG consists of four easily accessible, short, yet challenging puzzle games that have been developed to measure working memory (“Conveyer Belt”), visuospatial short-term memory (“Sunshine”), episodic recognition memory (“Viewpoint”), and planning (“Papyrinth”). A total of 641 participants were recruited for this study. Of these, 397 adults, 40 years and older (mean 54.9, SD 9.6), were eligible for analysis. Study participants played all games three times with 14 days in between sets. Face and content validity were based on expert opinion. Alternate forms reliability (AFR) was measured by comparing scores on different versions of the BAM-COG and expressed with an intraclass correlation (ICC: two-way mixed; consistency at 95%). Convergent validity (CV) was provided by comparing BAM-COG scores to gold-standard paper-and-pencil and computer-assisted cognitive assessment. Divergent validity (DV) was measured by comparing BAM-COG scores to the National Adult Reading Test IQ (NART-IQ) estimate. Both CV and DV are expressed as Spearman rho correlation coefficients.ResultsThree out of four games showed adequate results on AFR, CV, and DV measures. The games Conveyer Belt, Sunshine, and Papyrinth have AFR ICCs of .420, .426, and .645 respectively. Also, these games had good to very good CV correlations: rho=.577 (P=.001), rho=.669 (P<.001), and rho=.400 (P=.04), respectively. Last, as expected, DV correlations were low: rho=−.029 (P=.44), rho=−.029 (P=.45), and rho=−.134 (P=.28) respectively. The game Viewpoint provided less desirable results with an AFR ICC of .167, CV rho=.202 (P=.15), and DV rho=−.162 (P=.21).ConclusionsThis study provides evidence for the use of the BAM-COG test battery as a feasible, reliable, and valid tool to monitor cognitive performance in healthy adults in an online setting. Three out of four games have good psychometric characteristics to measure working memory, visuospatial short-term memory, and planning capacity.

Physical Activity and Cognitive Function of Long-Distance Walkers: Studying Four Days Marches Participants

“…Cognitive function was assessed using the Brain Aging Monitor -Cognitive Assessment (BAMCOG), which consists of online puzzle games that assess the performance in three cognitive domains: working memory, executive function, and visuospatial short-term memory (15,16). Previously, the BAMCOG was shown to have convergent validity with neuropsychological tests (Table 1 provides an overview of the puzzle games and associations with specific neuropsychological tests).…”

Peer Review #1 of "Episodix: a serious game to detect cognitive impairment in senior adults. A psychometric study (v0.1)"

“…Besides, after a thorough revision of the scientific literature in this field (Helkala et al, 1989;Perry, Watson & Hodges, 2000;Whitwell et al, 2007;Werner et al, 2009;Plancher et al, 2010;Grambaite et al, 2011;Libon et al, 2011;Raspelli et al, 2011;Aalbers et al, 2013;Tarnanas et al, 2013;Nolin et al, 2013;Fukui et al, 2015;Kawahara et al, 2015), included in our previous methodological review (Valladares-Rodriguez et al, 2016), it became apparent that more research was needed to obtain a robust psychometric validation that would eventually support the introduction of cognitive games such as Episodix, as a diagnostic tool in everyday clinical practice. In other words, there is a need to empirically evaluate the claim that serious games are a valid means to assess constructs and behaviors with at least the same validity than traditional approaches (Kato & de Klerk, 2017).…”

Design process and preliminary psychometric study of a video game to detect cognitive impairment in senior adults

“…The works referenced tackle a specific aspect of the broader topic of the introduction of games in neuroscience to detect mental disorders such as Mild Cognitive Impairment (MCI) (Nolin et al, 2013; Raspelli et al, 2011; Werner et al, 2009; Aalbers et al, 2013; Tarnanas et al, 2013; Kawahara et al, 2015; Fukui et al, 2015), Alzheimer’s Disease (AD) (Tarnanas et al, 2013; Kawahara et al, 2015; Fukui et al, 2015), Traumatic Brain Injury (TBI) (Wilson et al, 1989; Canty et al, 2014), Chronic Fatigue Syndrome (CFS) (Attree, Dancey & Pope, 2009), or Attention Deficit Hyperactivity Disorder (ADHD) (Parsons et al, 2007; Pollak et al, 2009). …”