The Oxford digital multiple errands test (OxMET): Validation of a simplified computer tablet based multiple errands test

Abstract

Impairments in executive functioning are common following Acquired Brain Injury, though there are few screening tools which present a time efficient and ecologically valid approach to assessing the consequences of executive impairments. We present the Oxford Digital Multiple Errands Test (OxMET), a novel and simplified computer-tablet version of a Multiple Errands Test.We recruited 124 neurologically healthy controls and 105 stroke survivors to complete the OxMET task. Normative data and internal consistency were established from the healthy control data. Convergent and divergent validation was assessed in a mixed subset of 158 participants who completed the OxMET and OCS-Plus. Test-retest reliability was examined across a mixed subset of 39 participants. Finally, we investigated the known-group discriminability of the OxMET.The OxMET demonstrated very high internal consistency, and stable group level testretest performance as well as good convergent and divergent validity. The OxMET demonstrated high sensitivity and good specificity in overall differentiation of stroke survivors from controls.The Oxford Digital Multiple Errands Test is a brief, easy to administer tool, designed to quickly screen for potential consequences of executive impairments in a virtual environment shopping task on a computer tablet. Initial normative data and validation within a chronic stroke cohort is presented.

Using Multiverse Analysis to Highlight Differences in Convergent Correlation Outcomes Due to Data Analytical and Study Design Choices

“…We used data from the OxMET psychometric investigation ( Webb et al, 2021 ). The OxMET, described in detail in Webb et al (2021) , is a simplified shopping scenario computer–tablet-based errands task.…”

“…We used data from the OxMET psychometric investigation ( Webb et al, 2021 ). The OxMET, described in detail in Webb et al (2021) , is a simplified shopping scenario computer–tablet-based errands task. A single score on overall accuracy is the primary outcome, though full performance metrics including detailed error related data are also calculated automatically and stored in the task.…”

“…A single score on overall accuracy is the primary outcome, though full performance metrics including detailed error related data are also calculated automatically and stored in the task. These include the number of rules broken, the frequency of rule breaks, perseverative errors, commissions, omissions, and total errors ( Webb et al, 2021 ).…”

“…The focus of this article is on convergent validity as one of the most common methods to investigate the validity of neuropsychological assessment in general ( Pawlowski et al, 2013 ), and particularly so with regards to the example of a Multiple Errands Tasks we will present here ( Webb et al, 2021 ). The 2014 Standards for Educational and Psychological Testing (Standards 1.12 and1.16) state that if a cognitive process is being assessed via a test, in this case executive functioning, then evidence must be generated that the test does assess that cognitive process ( American Educational Research Association et al, 2014 ), and to generate evidence using well construct-validated tools.…”

“…In the present multiverse demonstration, we, therefore, chose to focus on (a) sample group used, (b) sample size, (c) metrics used for outcome measures, and (d) covariate inclusion while acknowledging there are many potential other aspects that could affect validation outcomes. We use data from a recently developed multiple errands-based test, the Oxford Digital Multiple Errands Test (OxMET; Webb et al, 2021 ), which involved comparing the OxMET to two executive functioning tests (i.e., TMT and rule finding test), taken from the recently validated OCS-Plus (see Demeyere et al, 2021 for healthy control validation, and Webb et al, 2022 for subacute and chronic stroke validation).…”