Rethinking gesture phases: Articulatory features of gestural movement?

Bressem, Jana; Ladewig, Silva H.

doi:10.1515/semi.2011.022

Cited by 42 publications

(27 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Recent research e.g., in the ToGoG project has provided evidence that a number of gestures have undergone a process of grammaticalization (Bressem and Ladewig, 2011; Schoonjans, 2013). There is also psychological evidence that such information is immediately integrated (e.g., Ozyurek et al, 2007).…”

Section: Interactional Aspects Of Communication Already Accepted Amentioning

confidence: 99%

Grammar Is a System That Characterizes Talk in Interaction

Ginzburg

Poesio

2016

Front. Psychol.

View full text Add to dashboard Cite

Much of contemporary mainstream formal grammar theory is unable to provide analyses for language as it occurs in actual spoken interaction. Its analyses are developed for a cleaned up version of language which omits the disfluencies, non-sentential utterances, gestures, and many other phenomena that are ubiquitous in spoken language. Using evidence from linguistics, conversation analysis, multimodal communication, psychology, language acquisition, and neuroscience, we show these aspects of language use are rule governed in much the same way as phenomena captured by conventional grammars. Furthermore, we argue that over the past few years some of the tools required to provide a precise characterizations of such phenomena have begun to emerge in theoretical and computational linguistics; hence, there is no reason for treating them as “second class citizens” other than pre-theoretical assumptions about what should fall under the purview of grammar. Finally, we suggest that grammar formalisms covering such phenomena would provide a better foundation not just for linguistic analysis of face-to-face interaction, but also for sister disciplines, such as research on spoken dialogue systems and/or psychological work on language acquisition.

show abstract

Section: Interactional Aspects Of Communication Already Accepted Amentioning

confidence: 99%

Grammar Is a System That Characterizes Talk in Interaction

Ginzburg

Poesio

2016

Front. Psychol.

View full text Add to dashboard Cite

show abstract

“…The gesture annotation was identical to the one employed by Hoetjes et al (2015), and relied on the gesture phases distinguished by Kendon (1980Kendon ( , 2004, see e.g., also McNeill (1992), Bressem and Ladewig (2011) and Wagner et al (2014). According to this view, gesture production consists of a number of phases.…”

Section: Discussionmentioning

confidence: 99%

On what happens in gesture when communication is unsuccessful

Hoetjes

Krahmer

Swerts

2015

Speech Communication

View full text Add to dashboard Cite

On what happens in gesture when communication is unsuccessfulHoetjes, Marieke; Krahmer, Emiel; Swerts, Marc General rightsCopyright and moral rights for the publications made accessible in the public portal are retained by the authors and/or other copyright owners and it is a condition of accessing publications that users recognise and abide by the legal requirements associated with these rights.-Users may download and print one copy of any publication from the public portal for the purpose of private study or research -You may not further distribute the material or use it for any profit-making activity or commercial gain -You may freely distribute the URL identifying the publication in the public portal Take down policyIf you believe that this document breaches copyright, please contact us providing details, and we will remove access to the work immediately and investigate your claim. AbstractPrevious studies found that repeated references in successful communication are often reduced, not only at the acoustic level, but also in terms of words and manual co-speech gestures. In the present study, we investigated whether repeated references are still reduced in a situation when reduction would not be beneficial for the communicative situation, namely after the speaker receives negative feedback from the addressee. In a director-matcher task (experiment I), we studied gesture rate, as well as the general form of the gestures produced in initial and repeated references. In a separate experiment (experiment II) we studied whether there might (also) be more gradual differences in gesture form between gestures in initial and repeated references, by asking human judges which of two gestures (one from an initial and one from a repeated reference following negative feedback) they considered more precise. In both experiments, mutual visibility was added as a between subjects factor. Results showed that after negative feedback, gesture rate increased in a marginally significant way. With regard to gesture form, we found little evidence for changes in gesture form after negative feedback, except for a marginally significant increase of the number of repeated strokes within a gesture. Lack of mutual visibility only had a significant reducing effect on gesture size, and did not interact with repetition in any way. However, we did find gradual differences in gesture form, with gestures produced after negative feedback being judged as marginally more precise than initial gestures. The results from the present study suggest that in the production of unsuccessful repeated references, a process different from the reduction process as found in previous studies in repeated references takes place, with speakers appearing to put more effort into their gestures after negative feedback, as suggested by the data trending towards an increased gesture rate and towards gestures being judged as more precise after feedback.

show abstract

“…Focusing on gestures performed by a coordinated motion of hands and arms, it is also of interest to briefly review the common properties of their dynamics. More specifically, the following five phases have been identified in gesture formation [Bressem andLadewig 2011, Wagner et al 2014]:…”

Section: Multimodal Communication and Gesturesmentioning

confidence: 99%

Multimodal gesture recognition

Katsamanis¹,

Pitsikalis²,

Theodorakis³

et al. 2017

The Handbook of Multimodal-Multisensor Interfaces: Foundations, User Modeling, and Common Modality Combinations - Volume 1

View full text Add to dashboard Cite

2 Chapter 11 Multimodal Gesture Recognition Figure 11.1 Screenshot from the "Put That There!" demonstration video by the Architecture Machine Group at MIT [Bolt 1980]. ers [Kinect 2016]. Such advancements have led to intensified efforts to integrate multimodal gesture interfaces in real-life applications. Indeed, the field of multimodal gesture recognition has been attracting increasing interest, being driven by novel HCI paradigms on a continuously expanding range of devices equipped with multimodal sensors and ever-increasing computational power, for example smartphones and smart television sets. Nevertheless, the capabilities of modern multimodal gesture systems remain limited. In particular, the set of gestures accounted for in typical setups is mostly constrained to pointing gestures, a number of emblematic ones like an open palm, and gestures corresponding to some sort of interaction with a physical object, e.g., pinching for zooming. At the same time, fusion with speech remains in most cases just an experimental feature. When compared to the abundance and variety of gestures and their interaction with speech in natural human communication, it clearly seems that there is still a long way to go for the corresponding HCI research and development [Kopp 2013]. Multimodal gesture recognition constitutes a wide multidisciplinary field. This chapter makes an effort to provide a comprehensive overview of it, both in theoretical and application terms. More specifically, basic concepts related to gesturing, the multifaceted interplay of gestures and speech, and the importance of gestures in HCI are discussed in Section 11.2. An overview of the current trends in the field of multimodal gesture recognition is provided in Section 11.3, separately focusing on gestures, speech, and multimodal fusion. Further, a stateof-the-art recognition setup developed by the authors is described in detail in Section 11.4, in order to facilitate a better understanding of all practical considerations involved in such a system. In closing, the future of multimodal gesture recognition and related challenges are discussed in Section 11.5. Finally, a set of Focus Questions to aid comprehension of the material is also provided. 11.2 Multimodal Communication and Gestures 3 Glossary Terminology for Understanding Multimodal Gesture Recognition Co-speech gestures are gestures produced while speaking. Their interplay with speech and their role in human interaction are discussed in Section 11.2.2. Gesture recognition models are primarily statistical constructs that can be trained to represent specific gestures based on corresponding data. In this chapter, traditional hidden Markov models (HMMs) with Gaussian mixture model (GMM) observation probabilities are considered, as part of the authors' gesture recognition system detailed in Section 11.4. Further, a number of models based on deep learning approaches are overviewed in Section 11.3.1. A more elaborate discussion of deep learning for multimodal interaction modeling can be found in the second vol...

show abstract

Rethinking gesture phases: Articulatory features of gestural movement?

Cited by 42 publications

References 11 publications

Grammar Is a System That Characterizes Talk in Interaction

Grammar Is a System That Characterizes Talk in Interaction

On what happens in gesture when communication is unsuccessful

Multimodal gesture recognition

Contact Info

Product

Resources

About