Further Development of the Assessment of Military Multitasking Performance: Iterative Reliability Testing

Abstract

The Assessment of Military Multitasking Performance (AMMP) is a battery of functional dual-tasks and multitasks based on military activities that target known sensorimotor, cognitive, and exertional vulnerabilities after concussion/mild traumatic brain injury (mTBI). The AMMP was developed to help address known limitations in post concussive return to duty assessment and decision making. Once validated, the AMMP is intended for use in combination with other metrics to inform duty-readiness decisions in Active Duty Service Members following concussion. This study used an iterative process of repeated interrater reliability testing and feasibility feedback to drive modifications to the 9 tasks of the original AMMP which resulted in a final version of 6 tasks with metrics that demonstrated clinically acceptable ICCs of > 0.92 (range of 0.92–1.0) for the 3 dual tasks and > 0.87 (range 0.87–1.0) for the metrics of the 3 multitasks. Three metrics involved in recording subject errors across 2 tasks did not achieve ICCs above 0.85 set apriori for multitasks (0.64) and above 0.90 set for dual-tasks (0.77 and 0.86) and were not used for further analysis. This iterative process involved 3 phases of testing with between 13 and 26 subjects, ages 18–42 years, tested in each phase from a combined cohort of healthy controls and Service Members with mTBI. Study findings support continued validation of this assessment tool to provide rehabilitation clinicians further return to duty assessment methods robust to ceiling effects with strong face validity to injured Warriors and their leaders.

Front Desk Duty Multitasking Test After Mild Stroke: Preliminary Reliability and Validity

“…Preliminary validation results of the FDDT are similar to that of CQDT (Radomski et al, 2018;Weightman et al, 2017) and Complex Task Performance Assessment (CTPA), two other multitasking tests that use the structure and theoretical framework of MET (Shallice & Burgess, 1991). As with the CQDT and CTPA, results show that the FDDT can be scored reliably.…”

“…Using the scenario of a mock shift of overseeing the army barracks that can flexibly be set up in clinical and testing spaces, the CQDT was designed to place demands on multitasking as defined by Burgess (2000). The CQDT has been found to have acceptable levels of inter-rater reliability with interclass correlation coefficients for subscores ranging from 0.88 to 0.998 (Weightman et al, 2017). Scores on the CQDT were significantly different for groups of community controls and soldiers with mild traumatic brain injury on task accuracy and number of times that participants visited work areas to complete test tasks (Radomski et al, 2018).…”

Toward Return to Duty Decision-Making After Military Mild Traumatic Brain Injury: Preliminary Validation of the Charge of Quarters Duty Test

“…Inter-rater reliability of CQDT sub-scores were clinically acceptable with demonstrated intraclass correlations ranging from 0.88 to 1.0. 26 Although all participants were likely familiar with the task context (performing charge of quarters duty), the CQDT represented a novel task because of the specific assignment requirements and the challenge to complete the assignments in as few visits to work stations as possible.…”

“…23 The research team conducted a proof-of-concept study to examine the inter-rater reliability and preliminary validity of the six AMMP subtests. (See a complete description of iterative inter-rater reliability testing in Weightman et al 26 ) Here, we report on the preliminary validation results of the CQDT. Discriminant/known-group validity was evaluated by comparing the CQDT sub-scores of two groups of SM: (1) SM who were not duty-ready due to ongoing mTBI symptomatology and (2) SM considered to be healthy and duty-ready.…”

A novel assessment for Readiness Evaluation during Simulated Dismounted Operations: A reliability study

“…The AMMP was designed to assess function and inform duty-readiness decisions in SMs who suffered traumatic brain injury. [5] While the AMMP has strong ecologic validity through the use of simulated military tasks, and has demonstrated clinically acceptable interrater reliability, it is only focused on the symptoms and deficits of a very specific TBI population. Thus, there is a need for a more generalized assessment that can be used to evaluate readiness and identify deficits across a broad range of domains and diagnoses.…”