Visual salience effects on speaker choices: Direct or indirect influences on linguistic processing?

Abstract

The effect of visual salience on speakers’ choices is investigated by contrasting the effects of both visual and linguistic manipulations on picture descriptions and eye movements. Two-character pictures were used, which can be described in one of two complementary ways (e.g., a cop chasing a robber can be described either from a chasing or from a fleeing perspective), and using simple actives or other alternative syntactic structures (e.g., “a robber is being chased by a cop”). The pictures were preceded by a verb priming one of the two perspectives and/or a preview of one of the two characters. The results show that the visual manipulation affects looks to the characters regardless of which perspective had been linguistically primed, but it only affects verbal descriptions in the absence of a linguistic prime. Linguistically priming one of the perspectives, in contrast, has a reliable effect on both looks to the characters and verbal descriptions. These results suggest that visual salience does not influence linguistic choices directly.

Occluding the face diminishes the conceptual accessibility of an animate agent

“…These results suggest that participants conceptually distinguish Body-and Hand-Agent events: that the latter are more patientoriented than the former. This interpretation supports the findings of Kuchinsky (2009), Vogels et al (2013a), Bock and Ferreira (2014) and Antón-Méndez (2017), who argue that differences in visual prominence affect not only lexical retrieval but also event construal (i.e. figure-ground assignment), and thereby conceptual accessibility.…”

“…Conceptual properties are not alone, however, in influencing conceptual accessibility: the linguistic and non-linguistic context of an event, as well as perceptual features of the event, can also play a role. For example, when participants describe pictures with two entities, if they have seen one of the entities beforehand, they are more likely to mention this entity as Subject (Antón-Méndez, 2017;Osgood, 1971;Prentice, 1967;Sridhar, 1988), perhaps because it has become familiar or Given in the discourse. Prat-Sala and Branigan (2000) find a similar effect by manipulating linguistic discourse: hearing a narrative that puts linguistic focus on one of the entities in a picture increases the likelihood that the focused entity will be mentioned as Subject in a subsequent description task.…”

“…A wealth of research demonstrates that visual perceptual factors affect Subject selection (Antón-Méndez, 2017;Baltaretu, Krahmer, van Wijk, & Maes, 2016;Coco & Keller, 2009;Coco & Keller, 2015;Flores d'Arcais, 1975;Forrest, 1996;Gleitman et al, 2007;Hwang & Kaiser, 2015;Kuchinsky, 2009;Myachykov, Garrod, & Scheepers, 2018;Myachykov, Thompson, Garrod, & Scheepers, 2012;Osgood, 1971;Osgood & Bock, 1977;Sridhar, 1988;Tomlin, 1997;Vogels et al, 2013a). For example, in Gleitman et al (2007), participants viewed illustrations with two animate entities, such as a cat licking a dog.…”

“…An alternate explanation is that visual prominence affects Subject selection by influencing conceptual accessibility, specifically, how an event is construed (Antón-Méndez, 2017;Bock & Ferreira, 2014;Kuchinsky, 2009;Vogels et al, 2013a). Any single event in the world can be construed in multiple ways; for example, an event of one army running away from another can be described as either chasing or fleeing (DeLancey, 1991;Fisher, Hall, Rakowitz, & Gleitman, 1994).…”

“…We use Subject choice as a measure of conceptual accessibility; that is, which entity is mentioned as Subject in a language production task, and how sentences with different Subjects are interpreted in a language comprehension task. The majority of studies demonstrating perceptual effects on sentence formulation involve two entities that are matched for animacy (Antón-Méndez, 2017;Gleitman et al, 2007;Hwang & Kaiser, 2015;Myachykov, Thompson, Garrod, & Scheepers, 2012Tomlin, 1997;Vogels et al, 2013a; see also Griffin & Bock, 2000), perhaps reflecting a tacit assumption that perceptual influences will only reveal themselves when animacy is controlled and thus not a factor. To address our question, we selected an overt manipulation of visual prominence: partial occlusion of one of the referents.…”