Gestural Viewpoint Signals Referent Accessibility

Debreslioska, Sandra; Özyürek, Aslı; Gullberg, Marianne; Perniss, Pamela M.

doi:10.1080/0163853x.2013.824286

Cited by 58 publications

(44 citation statements)

References 33 publications

(63 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Speakers point to the entities they refer to with speech, use iconic gestures as they move the fingers of an inverted V-hand in a wiggling manner while saying 'he walked across', use bodily demonstrations of reported actions as they tell narratives, convey different viewpoints of events or use gesture spaces indexing different levels of discourse cohesion parallel to marking similar discourse devices found in speech (e.g. [10,11]). Thus, there has been mounting evidence at the production level that cospeech gestures contribute semantic, syntactic, discursive and pragmatic information to the verbal part of an utterance, forming composite utterances with semiotic diversity [6][7][8]12].…”

Section: Introductionmentioning

confidence: 99%

Hearing and seeing meaning in speech and gesture: insights from brain and behaviour

Özyürek

2014

Phil. Trans. R. Soc. B

146

View full text Add to dashboard Cite

As we speak, we use not only the arbitrary form–meaning mappings of the speech channel but also motivated form–meaning correspondences, i.e. iconic gestures that accompany speech (e.g. inverted V-shaped hand wiggling across gesture space to demonstrate walking). This article reviews what we know about processing of semantic information from speech and iconic gestures in spoken languages during comprehension of such composite utterances. Several studies have shown that comprehension of iconic gestures involves brain activations known to be involved in semantic processing of speech: i.e. modulation of the electrophysiological recording component N400, which is sensitive to the ease of semantic integration of a word to previous context, and recruitment of the left-lateralized frontal–posterior temporal network (left inferior frontal gyrus (IFG), medial temporal gyrus (MTG) and superior temporal gyrus/sulcus (STG/S)). Furthermore, we integrate the information coming from both channels recruiting brain areas such as left IFG, posterior superior temporal sulcus (STS)/MTG and even motor cortex. Finally, this integration is flexible: the temporal synchrony between the iconic gesture and the speech segment, as well as the perceived communicative intent of the speaker, modulate the integration process. Whether these findings are special to gestures or are shared with actions or other visual accompaniments to speech (e.g. lips) or other visual symbols such as pictures are discussed, as well as the implications for a multimodal view of language.

show abstract

Section: Introductionmentioning

confidence: 99%

Hearing and seeing meaning in speech and gesture: insights from brain and behaviour

Özyürek

2014

Phil. Trans. R. Soc. B

146

View full text Add to dashboard Cite

show abstract

“…Empirical studies on reference tacking in various languages seem to support this account. For example, Debreslioska et al (2013) found that German adult native speakers use mainly nominal forms with less accessible referents while they prefer attenuated form with more accessible referents. In the following section we provide a short overview of linguistic forms available in Turkish and previous literature on reference tracking in speech in Turkish.…”

Section: Reference Tracking In Speechmentioning

confidence: 98%

“…That is they use the visual modality to maintain discourse cohesion, i.e., to mark the accessibility of the referents in discourse (Debreslioska, Özyürek, Gullberg, & Perniss, 2013;Gullberg, 2006;Perniss & Özyürek, 2015;Yoshioko, 2008). Studies suggest that similar to speech, co-speech gestures, the meaningful movements of the hands that accompany speech, are sensitive to referential context in terms of the quantity of marking material (Gullberg, 2003(Gullberg, , 2006Levy & Fowler 2000;McNeill & Levy, 1993).…”

Section: Introductionmentioning

confidence: 98%

Discourse management

Azar

Özyürek

2015

DuJAL

View full text Add to dashboard Cite

Speakers achieve coherence in discourse by alternating between differential lexical forms e.g. noun phrase, pronoun, and null form in accordance with the accessibility of the entities they refer to, i.e. whether they introduce an entity into discourse for the first time or continue referring to an entity they already mentioned before. Moreover, tracking of entities in discourse is a multimodal phenomenon. Studies show that speakers are sensitive to the informational structure of discourse and use fuller forms (e.g. full noun phrases) in speech and gesture more when re-introducing an entity while they use attenuated forms (e.g. pronouns) in speech and gesture less when maintaining a referent. However, those studies focus mainly on non-pro-drop languages (e.g. English, German and French). The present study investigates whether the same pattern holds for pro-drop languages. It draws data from adult native speakers of Turkish using elicited narratives. We find that Turkish speakers mostly use fuller forms to code subject referents in re-introduction context and the null form in maintenance context and they point to gesture space for referents more in re-introduction context compared maintained context. Hence we provide supportive evidence for the reverse correlation between the accessibility of a discourse referent and its coding in speech and gesture. We also find that, as a novel contribution, third person pronoun is used in re-introduction context only when the referent was previously mentioned as the object argument of the immediately preceding clause.

show abstract

“…It is important to note, however, that because the criterion for categorizing a gestures as having an observer viewpoint is that a referent is represented as a whole on the forelimbs (e.g., McNeill, 1992McNeill, , 2005Parrill, 2009;Debreslioska et al, 2013), the difference between character and observer viewpoint in these studies essentially boils down to what happens on the hands.…”

Section: Describing Gestur Al Elementsmentioning

confidence: 99%

“…In Figure 1a, the speaker shows how a character in her narrative acts by using her own body as Figure 1b the speaker uses only her right hand to show the actions of a character, that is, she uses her hand to show the entity as a whole. In the gesture literature, this difference has generally been referred to as a difference in viewpoint (McNeill, 1992(McNeill, , 2005Beattie & Shovelton, 2001Parrill, 2009, 2010: Stec, 2012Debreslioska, Özyürek, Gullberg, & Perniss, 2013) -most often framed as c h a r a c t e r v i e w p o i n t versus o b s e r v e r v i e w p o i n t, although studies sometimes use different labels for comparable distinctions. 2 Although the term viewpoint is often used in the co-speech gesture literature, exactly what constitutes a viewpoint is rarely specified.…”

Section: Viewpoint In Gesturementioning

confidence: 99%

Separating viewpoint from mode of representation in iconic co-speech gestures: insights from Danish narratives

Frederiksen¹

2017

Lang. cogn.

View full text Add to dashboard Cite

a b st r a c t During narrative retelling, speakers shift between different viewpoints to reflect how they conceptualize the events that unfolded. These viewpoints can be indicated through gestural means as well as through verbal ones. Studies of co-speech gestures have inferred viewpoint from gesture form, i.e., how entities are mapped onto the (primarily manual) articulators, but the merits of this approach have not been discussed. The present study argues that viewpoint is more than gestural form. Despite connections between the two, many other factors may influence a gesture's form. Assessing viewpoint from gesture form alone limits the applicability of gestural viewpoint as a window onto speakers' event conceptualization and introduces unnecessary differences in the categorization of viewpoint across gestures types. The present study examines iconic co-speech gestures in Danish narratives, and makes explicit the means used to infer gestural viewpoint. The approach advocated here ensures that the notion of viewpoint can be applied in a principled way to all or most iconic gestures.k e y w o r d s : viewpoint, gesture, sign language, role-shift, mode of representation.

show abstract

Gestural Viewpoint Signals Referent Accessibility

Cited by 58 publications

References 33 publications

Hearing and seeing meaning in speech and gesture: insights from brain and behaviour

Hearing and seeing meaning in speech and gesture: insights from brain and behaviour

Discourse management

Separating viewpoint from mode of representation in iconic co-speech gestures: insights from Danish narratives

Contact Info

Product

Resources

About