Development of a new virtual reality test of cognition: assessing the test-retest reliability, convergent and ecological validity of CONVIRT

Abstract

Background: Technological advances provide an opportunity to refine tools that assess central nervous system performance. This study aimed to assess the test-retest reliability and convergent and ecological validity of a newly developed, virtual-reality, concussion assessment tool, 'CONVIRT', which uses eye-tracking technology to assess visual processing speed, and manual reaction time (pushing a button on a riding crop) to assess attention and decision-making. CONVIRT was developed for horse jockeys, as of all sportspersons, they are most at risk of concussion. Methods: Participants (N = 165), were assessed with CONVIRT, which uses virtual reality to give the user the experience of riding a horse during a horserace. Participants were also assessed with standard Cogstate computer-based concussion measures in-between two completions of the CONVIRT battery. The physiological arousal induced by the test batteries were assessed via measures of heart rate and heart rate variability (LF/HF ratio). Results: Satisfactory test-retest reliability and convergent validity with Cogstate attention and decision-making subtests and divergent validity in visual processing speed measures were observed. CONVIRT also increased heart rate and LF/HF ratio, which may better approximate participant arousal levels in their workplace. Conclusions: CONVIRT may be a reliable and valid tool to assess elements of cognition and CNS disruption. The increased ecological validity may also mean better informed 'return-to-play' decisions and stronger industry acceptance due to the real-world meaningfulness of the assessment. However, before this can be achieved, the sensitivity of the CONVIRT battery needs to be demonstrated.

Using Medical Device Standards for Design and Risk Management of Immersive Virtual Reality for At-Home Therapy and Remote Patient Monitoring

“…The sense of presence that immersive VR offers has considerable potential to differentiate the impact of VR in clinical contexts, including telerehabilitation, moving forward [17][18][19]. Critically, immersive VR offers the potential for greater ecological validity in therapy, allowing the brain to respond to stimuli similar to how it does in the real world [20][21][22][23][24].…”

Assessing the utility of a virtual-reality neuropsychological test battery, ‘CONVIRT’, in detecting alcohol-induced cognitive impairment

“…The eye tracking unit embedded in the FOVE HMD has a tracking accuracy of less than 1 degree and a refresh rate of 120 Hz (https://www.getfove.com).Through the HMD, participants experience the first-person perspective of a jockey riding a horse in a horserace. Full specifications of the Gigabyte P35 laptop, FOVE 0 HMD, and CONVIRT VR application are reported previously (Horan et al, 2020).…”

“…In line with our earlier work (Horan et al, 2020), a composite score (COMP) was computed for cognitive performance on the three CONVIRT tests (i.e., SAC-VR, DET-VR, IDN-VR). The COMP was computed for each individual by averaging the standardised score of the three tests.…”

“…The CONVIRT battery was developed within a VR framework to assess and determine if the cognitive abilities of a concussed horse jockey had returned to pre-concussion levels. The CONVIRT battery uses manual simple and choice reaction time, and saccadic reaction time to assess components of attention, decision-making, and visual processing speed within a dynamic VR environment where these cognitive operations are assessed in the context of a virtual horse race (see Horan et al, 2020 for a full description). The greatest threats to the health of professional jockeys are concussion and head injury.…”

“…In a large study comparing the CONVIRT battery with validated and comparable tests from the computerised Cogstate battery, (N = 165), the CONVIRT battery of virtual-reality cognitive tests showed excellent test-retest reliability, and convergent validity with the Cogstate attention and decision-making tasks (identification and detection tasks; Horan et al, 2020). Evidence of greater physiological arousal during performance from measures of heart rate and heart rate variability also suggested that performance on the CONVIRT battery was more ecologically valid than the Cogstate battery.…”