Eye movements as a window into real-time spoken language comprehension in natural contexts

Abstract

When listeners follow spoken instructions to manipulate real objects, their eye movements to the objects are closely time locked to the referring words. We review five experiments showing that this time-locked characteristic of eye movements provides a detailed profile of the processes that underlie real-time spoken language comprehension. Together, the first four experiments showed that listeners immediately integrated lexical, sublexical, and prosodic information in the spoken input with information from the visual context to reduce the set of referents to the intended one. The fifth experiment demonstrated that a visual referential context affected the initial structuring of the linguistic input, eliminating even strong syntactic preferences that result in clear garden paths when the referential context is introduced linguistically. We argue that context affected the earliest moments of language processing because it was highly accessible and relevant to the behavioral goals of the listener.

Processing the presence, placement, and properties of a distractor in spatial language tasks

“…Such an assumption is consistent with work in communication that argues that referential descriptions that include distinct attributes that make an object stand out in contrast to surrounding objects are particularly helpful in disambiguating the intended referent (Brown-Schmidt, Byron, & Tanenhaus, 2005;Eberhard, Spivey-Knowlton, Sedivy, & Tanenhaus, 1995;Olson, 1970). For example, Olson argued that a speaker would describe the same small round white block as the white one when in the context of a small round black block, and as the round one when in the context of a small square white block.…”

“…However, one might imagine that in tasks in which there is a different goal (e.g., identification or discrimination), different dimensions may be prioritized. Olson's (1970) changing description of the small round white block as a function of its context is a good example of the prioritization of other dimensions on the basis of task or context (see also Brown-Schmidt et al, 2005;Eberhard et al, 1995). Second, spatial features are not strictly independent of other features of an object.…”

Online interpretation of scalar quantifiers: Insight into the semantics–pragmatics interface

“…Our first analysis examined a coarse-grain measure of subjects' use of the quantifier during five periods of analysis: All time windows began and ended 200 ms after the relevant marker in the speech stream to account for the time it would take to program saccadic eye-movements (Eberhard et al, 1995). For each trial, we summed the total number of looks to the target character and gender distracter within each of these intervals and calculated the proportion of looks to the target over looks to both.…”

“…The visual-world eye-tracking paradigm has been used extensively in psycholinguistic research to yield a sensitive, time-locked measure of linguistic processing (Eberhard et al, 1995). Participants are presented with spoken instructions, asking them to manipulate objects within a visual reference world, while their eye-movements to those objects are measured.…”

Viewing and naming objects: eye movements during noun phrase production

“…Tanenhaus et al (1995Tanenhaus et al ( , 1996; see also Eberhard et al, 1995) registered eye movements to study spoken language comprehension. They found that the eye movements of listeners carrying out spoken instructions, such as 'put the wallet above the candle', were tightly time-locked to the speech input and depended on a number of variables known to affect the ease of spoken language understanding.…”