On the give and take between event apprehension and utterance formulation

Abstract

Two experiments are reported which examine how manipulations of visual attention affect speakers' linguistic choices regarding word order, verb use and syntactic structure when describing simple pictured scenes. Experiment 1 presented participants with scenes designed to elicit the use of a perspective predicate (The man chases the dog/The dog flees from the man) or a conjoined noun phrase sentential Subject (A cat and a dog/A dog and a cat). Gaze was directed to a particular scene character by way of an attention-capture manipulation. Attention capture increased the likelihood that this character would be the sentential Subject and altered the choice of perspective verb or word order within conjoined NP Subjects accordingly. Experiment 2 extended these results to word order choice within Active versus Passive structures (The girl is kicking the boy/The boy is being kicked by the girl) and symmetrical predicates (The girl is meeting the boy/The boy is meeting the girl). Experiment 2 also found that early endogenous shifts in attention influence word order choices. These findings indicate a reliable relationship between initial looking patterns and speaking patterns, reflecting considerable parallelism between the on-line apprehension of events and the on-line construction of descriptive utterances.

Integrating mechanisms of visual guidance in naturalistic language production

“…This definition of perceptual guidance is compatible with that of visual saliency (Itti & Koch, 2000;Parkhursta, Lawb, & Niebur, 2002). Saliency, as a composite measure of low-level visual information, is positively correlated with the presence of objects (Elazary & Itti, 2008), and found to guide attention in tasks where targets are underspecified (e.g., memorization, Underwood & Foulsham, 2006), and also utilized during situated language production (Gleitman et al, 2007). In our view, the most appropriate way to define visual saliency is in terms of visual clutter, calculated by integrating low-level visual information, e.g., color, with edge information (Rosenholtz, Mansfield, & Jin, 2005).…”

“…The interaction between visual attention and sentence processing could potentially take place in a number of ways, and a wide variety of factors have been identified, both linguistic (e.g., Meyer et al, 1998;Griffin & Bock, 2000;Bock et al, 2003;Kuchinksy et al, 2011) and perceptual (e.g., Gleitman et al, 2007;Papafragou et al, 2008;Myachykov et al, 2011). Language production is sensitive to simple perceptual events (e.g., a light flash at the target location, Gleitman et al, 2007), but also interacts with complex perceptual information, such as the type of motion depicted (influencing whether manner or path verbs are produced (Papafragou et al, 2008), or the complexity of the depicted event (influencing grammatical role assignment and linear position, Myachykov et al, 2011).…”

“…On one hand, there is evidence that perceptual factors are involved in selecting referential information for sentence processing (e.g., Arnold & Griffin, 2007;Gleitman, January, Nappa, & Trueswell, 2007;Papafragou, Hulbert, & Trueswell, 2008; see also Myachykov, Thompson, Scheepers, & Garrod, 2011, for a review). On the other hand, there is also evidence for sentence processing mechanisms which explain how eye-movements and speech responses are linked (Meyer, Sleiderink, & Levelt, 1998;Griffin & Bock, 2000;Bock, Irwin, Davidson, & Levelt, 2003;Qu & Chai, 2008;Kuchinksy, Bock, & Irwin, 2011).…”

“…Similarly, although perceptual guidance is generally taken to refer to low-level properties of the visual information (Gleitman et al, 2007), such definition is not always consistent across studies. For example, Kuchinksy et al (2011) use the term to refer to experience of reading clocks, which again reflects task-based, conceptual knowledge (short hand for hours, long hand for minutes).…”

Do as I say, not as I do: A lexical distributional account of English locative verb class acquisition

“…If these roles are not assigned, then a locative structure is inappropriate. Furthermore, it is thought that the relative salience of arguments can determine structural choices independently of the verb (Bock 1982;Gleitman, January, Nappa & Trueswell, 2007;Goldberg, 1995). Indeed, Gropen et al (1991a) demonstrated that changes in the salience of LOCATION and THEME at test did change Figure 1.…”