Test–Retest Reliability of Eye Tracking in the Visual World Paradigm for the Study of Real-Time Spoken Word Recognition

Abstract

Purpose Researchers have begun to use eye tracking in the visual world paradigm (VWP) to study clinical differences in language processing, but the reliability of such laboratory tests has rarely been assessed. This paper assesses test-retest reliability of the VWP for spoken word recognition. Methods Participants performed an auditory VWP task in repeated sessions and a visual-only VWP task in a third session. We performed correlation and regression analyses on several parameters to determine which reflect reliable behavior and which are predictive of behavior in later sessions. Results Results showed that the fixation parameters most closely related to timing and degree of fixations were moderately-to-strongly correlated across days, while the parameters related to rate of increase or decrease of fixations to particular items were less strongly correlated. Moreover, when including factors derived from the visual-only task, the performance of the regression model was at least moderately correlated with Day 2 performance on all parameters (R > .30). Conclusions The VWP is stable enough (with some caveats) to serve as an individual measure. These findings suggest guidelines for future use of the paradigm and for areas of improvement in both methodology and analysis.

Detecting time-specific differences between temporal nonlinear curves: Analyzing data from the visual world paradigm

“…More recent approaches 3,6 have used nonlinear functions designed to capture more meaningful aspects of variation in these time series. For example, target fixations can be fit by a four-parameter logistic function that captures the asymptotes, slope, and crossover point.…”

“…The functional form we use here has been employed successfully, 3,6 where it is termed the asymmetric Gaussian curve. The function essentially merges a Gaussian curve on the left with a Gaussian curve on the right that shares the same peak but have separate variances and baselines $$p_{it}=true\{\begin{array}{cc}left{B}_i-A_{1i})*\text{exp}(\frac{{(t-{\mu }_i)}^2}{-2{\sigma }_{1i}^2})+A_{1i}& left\text{if}t\le {\mu }_i\\ left(B_i-A_{2i})*\text{exp}\left(\frac{{(t-{\mu }_i)}^2}{-2{\sigma }_{2i}^2}\right)+A_{2i}& left\text{if}t>\mu \end{array}$$ Again, p it denotes the probability of looking at that object for individual i at time t .…”

“…These two functions have been successfully employed in a number of prior studies with this sort of data. 3,6,13 …”

“…5 We build this method upon recent work using nonlinear curves to describe such data. 3,6 To briefly summarize the need for such an approach, while standard statistical techniques are well equipped to detect an overall difference between time series, it is much harder to identify the specific time periods during which sufficient evidence exists to declare exactly where the difference occurs while maintaining statistical rigor.…”

An Open Conversation on Using Eye-Gaze Methods in Studies of Neurodevelopmental Disorders

“…These effects may be undesirable, difficult to control, or different across groups and should be assessed during the piloting process and reported in full if they occur. Additional studies are needed to investigate this issue (but see Farris-Trimble & McMurray, 2013;Farzin, Scaggs, Hervey, Berry-Kravis, & Hessl, 2011, for examples of reliability studies).…”