Methodological Problems With Online Concussion Testing

Abstract

Reaction time testing is widely used in online computerized concussion assessments, and most concussion studies utilizing the metric have demonstrated varying degrees of difference between concussed and non-concussed individuals. The problem with most of these online concussion assessments is that they predominantly rely on consumer grade technology. Typical administration of these reaction time tests involves presenting a visual stimulus on a computer monitor and prompting the test subject to respond as quickly as possible via keypad or computer mouse. However, inherent delays and variabilities are introduced to the reaction time measure by both computer and associated operating systems that the concussion assessment tool is installed on. The authors hypothesized systems that are typically used to collect concussion reaction time data would demonstrate significant errors in reaction time measurements. To remove human bias, a series of experiments was conducted robotically to assess timing errors introduced by reaction time tests under four different conditions. In the first condition, a visual reaction time test was conducted by flashing a visual stimulus on a computer monitor. Detection was via photodiode and mechanical response was delivered via computer mouse. The second condition employed a mobile device for the visual stimulus, and the mechanical response was delivered to the mobile device's touchscreen. The third condition simulated a tactile reaction time test, and mechanical response was delivered via computer mouse. The fourth condition also simulated a tactile reaction time test, but response was delivered to a dedicated device designed to store the interval between stimulus delivery and response, thus bypassing any problems hypothesized to be introduced by computer and/or computer software. There were significant differences in the range of responses recorded from the four different conditions with the reaction time collected from visual stimulus on a mobile device being the worst and the device with dedicated hardware designed for the task being the best. The results suggest that some of the commonly used visual tasks on consumer grade computers could be (and have been) introducing significant errors for reaction time testing and that dedicated hardware designed for the reaction time task is needed to minimize testing errors.

Chronic Neurophysiological Effects of Repeated Head Trauma in Retired Australian Male Sport Athletes

“…We have previously used cognitive testing batteries (e.g., CogState Brief Battery and Cambridge Neuropsychological Test Automated Battery) to compare retired athletes to age and education matched controls, with mixed results ( 4 , 11 ). With recent concerns regarding online cognitive testing programs ( 53 ), future research will incorporate more comprehensive sensitive and specific neuropsychological evaluations, rather than computerised batteries, alongside TMS and somatosensory testing, that can allow for more refined diagnostics and longitudinal monitoring of future decline as well as management outcomes.…”

Changes in processing speed during early abstinence from alcohol dependence

“…Processing speed can be assessed via ‘reaction time’ (RT), which can be ‘simple’ (one stimulus, one response) or ‘choice’ (more executive, usually with multiple potential stimuli each requiring a distinct response) ( Cepeda et al, 2013 ). Stimulus and response modality during RT assessments can affect results ( Guillot et al, 2010 ), and noise can be added via a number of technical elements ( Holden et al, 2020 ). RT assessed using vibrotactile stimuli (vibration through touch) reduces competing same-sense distractions ( Holden et al, 2020 ; Tommerdahl et al, 2016 ), and doing so using dedicated hardware and software (such as Brain Gauge Pro (Cortical Metrics, Chapel Hill, NC, USA, https://www.corticalmetrics.com ), used in the current study) gives the most accurate RT with the least variability ( Holden et al, 2019 ).…”

“…Stimulus and response modality during RT assessments can affect results ( Guillot et al, 2010 ), and noise can be added via a number of technical elements ( Holden et al, 2020 ). RT assessed using vibrotactile stimuli (vibration through touch) reduces competing same-sense distractions ( Holden et al, 2020 ; Tommerdahl et al, 2016 ), and doing so using dedicated hardware and software (such as Brain Gauge Pro (Cortical Metrics, Chapel Hill, NC, USA, https://www.corticalmetrics.com ), used in the current study) gives the most accurate RT with the least variability ( Holden et al, 2019 ). Finally, as the stimuli are not letters, numbers or words, this reduces the risk of educational bias, and while instructions are presented on-screen, the facilitator can additionally explain the task verbally.…”

Younger, drunk, and fast: Paradoxical rapid reaction time in hazardous drinkers

“…Firstly, the organisation of the somatosensory system is somatotopic (adjacent regions of the body represented adjacently), and is therefore ideal for inducing cortico–cortical interactions in adjacent or near-adjacent cortical regions ( Nelson and Chen, 2008 ). Secondly, compared to auditory or visual input, it is also easier to limit competing same-sense distractions ( Holden et al, 2020 ; Tommerdahl et al, 2016 ). With regard to RT assessment specifically, noise can be added by computer systems, core processors, screen refresh rates and other hardware/software processing latencies ( Holden et al, 2020 ).…”

“…Secondly, compared to auditory or visual input, it is also easier to limit competing same-sense distractions ( Holden et al, 2020 ; Tommerdahl et al, 2016 ). With regard to RT assessment specifically, noise can be added by computer systems, core processors, screen refresh rates and other hardware/software processing latencies ( Holden et al, 2020 ). Holden et al (2019) suggest that tactile stimulation using dedicated hardware is the most accurate method for RT assessment compared to visual stimuli with various response methods, and the one with the least RT variability.…”