Using video ratings to assess multitasking performance in a naturalistic paradigm

Abstract

BACKGROUND: Multitasking measures, in which a series of tasks must be completed within a naturalistic setting not fully under the experimenter's control, have been shown to be more sensitive than traditional measures in detecting organisational problems in people with difficulties in executive functioning. There are a number of drawbacks to such tasks however. They can take considerable time to administer and are demanding in terms of examiners noting and recording all relevant aspects of performance. This potentially leaves them more open to subtle bias. One method that could offset these limitations is to video record performance. OBJECTIVES: The practicality and outcome of using video ratings to accurately score performance off-line is investigated here. METHODS: Nineteen participants completed a Multiple Errands Task (MET) while wearing a body-worn camera. Their performance was scored "live" and by an independent rater who had only access to video footage of the task. RESULTS: Significant relationships were seen on all variables of the MET between the live and video ratings. The inter-rater reliability of the measure appears strong. CONCLUSION: We provide initial support for the use of a video rater when assessing performance on an MET.

The Multiple Errands Test: A Guide for Site-Specific Version Development

“…Key location staff were initially debriefed before each test, to prevent them being too helpful (Robnett et al, 2021) or intimidated by the clipboard (Hynes et al, 2014), but this was ceased due to low efficacy and large time burden.…”