We present our work on the temporal integration of hierarchies of communicative actions: kinesic, prosodic and discursive. We use the device of the ‘catchment’ as the locus around which this integration proceeds. We present a detailed case study of a gesture and speech elicitation experiment in which a subject describes her living space to an interlocutor. First, we process the video data to obtain the motion traces of both of the subject’s hands using the vector coherence mapping algorithm. We code the gestures to identify the catchments. We recover discourse purposes utilizing a system of guided questions. Finally, we define prosody in terms of the ToBI system. The results of these analyses are compared against the computed motion traces to identify the cues accessible in the gestural and audio data that correlate well with the psycholinguistic analyses. The results show that motion, prosody and discourse structure are integrated at each moment of speaking. The electronic edition of this article includes audio-visual data.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.
customersupport@researchsolutions.com
10624 S. Eastern Ave., Ste. A-614
Henderson, NV 89052, USA
This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.
Copyright © 2024 scite LLC. All rights reserved.
Made with 💙 for researchers
Part of the Research Solutions Family.