Coregistration of eye movements and EEG in natural reading: Analyses and review.

Abstract

Brain-electric correlates of reading have traditionally been studied with word-by-word presentation, a condition that eliminates important aspects of the normal reading process and precludes direct comparisons between neural activity and oculomotor behavior. In the present study, we investigated effects of word predictability on eye movements (EM) and fixation-related brain potentials (FRPs) during natural sentence reading. Electroencephalogram (EEG) and EM (via video-based eye tracking) were recorded simultaneously while subjects read heterogeneous German sentences, moving their eyes freely over the text. FRPs were time-locked to first-pass reading fixations and analyzed according to the cloze probability of the currently fixated word. We replicated robust effects of word predictability on EMs and the N400 component in FRPs. The data were then used to model the relation among fixation duration, gaze duration, and N400 amplitude, and to trace the time course of EEG effects relative to effects in EM behavior. In an extended Methodological Discussion section, we review 4 technical and data-analytical problems that need to be addressed when FRPs are recorded in free-viewing situations (such as reading, visual search, or scene perception) and propose solutions. Results suggest that EEG recordings during normal vision are feasible and useful to consolidate findings from EEG and eye-tracking studies.

Eye movements and brain electric potentials during reading

“…Here we review results from our research (1) from studies that have collected ERPs and EMs during reading the same sentences, but with different groups of subjects (Dambacher, Kliegl, Hofmann, & Jacobs, 2006;Dambacher & Kliegl, 2007), and (2) from an experiment during which ERPs and EMs were recorded simultaneously in a single group of subjects (Dimigen, Sommer, Hohlfeld, Jacobs, & Kliegl, 2011). Such co-registration research encounters quite a few technical and substantive problems, but also opportunities for a deeper understanding of the relation between oculomotor effects and brain signals (Dimigen et al, 2011). In all experiments subjects read the Potsdam Sentence Corpus (PSC) that comprises frequency and predictability norms for each of its 1138 words (Kliegl, Grabner, Rolfs, & Engbert, 2004).…”

“…We co-registered and analyzed EMs and ERPs for all words of the PSC in a natural, left-toright fashion while 30 subjects read at their individual pace (Dimigen et al, 2011). A 32-channel EEG recording system was linked to a high-resolution infrared eye tracker.…”

Stimulus onset asynchrony and the timeline of word recognition: Event-related potentials during sentence reading

“…This involves, for instance, self-paced reading (e.g., Ditman, Holcomb, & Kuperberg, 2007) or word-length dependent presentation durations in the RSVP paradigm (e.g., Nieuwland & Van Berkum, 2006;Nieuwland & Kuperberg, 2008) as well as the simultaneous recording of EEG and eye movements in normal left-to-right reading (e.g., Dimigen, Sommer, Hohlfeld, Jacobs, & Kliegl, 2011). Nevertheless, the conditions and extent of effects of presentation rate on the time course of word recognition are not well established.…”

Eye movements in children during reading: a review

“…Also, they cannot be used to investigate the role of para-foveal perception during reading. The timing and extent to which upcoming words are preprocessed is still controversial and can be studied in greater detail with EFRPs (Dimigen, Sommer, Hohlfeld, Jacobs, & Kliegl, 2011). For the time being, this technique is faced with specific methodological challenges, particularly due to effects of overlap between EEG responses elicited by successive fixations, that can interfere with effects resulting from experimental conditions.…”