Referential and Visual Cues to Structural Choice in Visually Situated Sentence Production

Abstract

We investigated how conceptually informative (referent preview) and conceptually uninformative (pointer to referent’s location) visual cues affect structural choice during production of English transitive sentences. Cueing the Agent or the Patient prior to presenting the target-event reliably predicted the likelihood of selecting this referent as the sentential Subject, triggering, correspondingly, the choice between active and passive voice. Importantly, there was no difference in the magnitude of the general Cueing effect between the informative and uninformative cueing conditions, suggesting that attentionally driven structural selection relies on a direct automatic mapping mechanism from attentional focus to the Subject’s position in a sentence. This mechanism is, therefore, independent of accessing conceptual, and possibly lexical, information about the cued referent provided by referent preview.

How language production shapes language form and comprehension

“…First, if the relationship between noun animacy and active/passive form is the result of utterance planning pulled between Plan Reuse (favoring the more common Active form) and Easy First (favoring early mention of animates), then we would expect that animacy/Easy First effects on structure would be smaller in those languages (such as Slavic languages) that have a strong bias to use active forms. Results of this type are perhaps not surprising, because by definition, a strong allegiance to a single dominant word order to convey a particular message will allow less room for word order flexibility to accommodate ease of retrieval (Myachykov et al, 2011; Gennari et al, 2012) 2 . Second, utterance planning time should increase when these forces conflict compared to situations when they converge on the same form.…”

Sensitivity to salience: linguistic vs. visual cues affect sentence processing and pronoun resolution

“…Speakers' eye-movements during sentence production substantiate that what speakers look at in depicted actions is what they mention first: Visual-attentive cues, even when the speaker is unaware of them, indicate a referent as more salient amongst others, and hence as earlier accessible and most likely to be mentioned first (e.g., Gleitman, January, Nappa, & Trueswell, 2007;Myachykov, Thompson, Garrod, & Scheepers, 2012;Tomlin, 1997; for a discussion of broader cognitive and communicative factors that affect first-mention see Bock, Irwin, & Davidson, 2004). In contrast to mental accessibility (e.g., Ariel, 1988), these production studies establish accessibility at the lemma/conceptual level (Bock & Warren, 1985).…”

“…Moreover, production studies, revealing an impact of visual cueing on firstmention, depicted referents in action such that theta roles were visible in the visual context (e.g., Gleitman et al, 2007;Myachykov et al, 2012). Therefore, we aimed to account for these two factors by controlling for ORIENTATION of animals in the visual scene (left-toright vs. right-to-left) as well as for the predictability of the THETA ROLE of the first-mentioned referent (To improve readability, the results regarding these two control factors are reported in the Supplemental data).…”

Interfaces between language and cognition

“…The results discussed by Roelofs and Piai (2011) demonstrate how gaze shifts can be linked to the process of phonological encoding with specific focus on word production automaticity. The article by Myachykov et al (2012) presents evidence about the special role attention plays in determining the assignment of grammatical roles and the associated syntactic choice in visually situated sentence production. Papers by Huettig et al (2012), Knoeferle et al (2011), and Kaiser (2012) provide complementary evidence about the involvement of the language-cognition interface during sentence comprehension in visually situated contexts.…”