We present our work on the temporal integration of hierarchies of communicative actions: kinesic, prosodic and discursive. We use the device of the ‘catchment’ as the locus around which this integration proceeds. We present a detailed case study of a gesture and speech elicitation experiment in which a subject describes her living space to an interlocutor. First, we process the video data to obtain the motion traces of both of the subject’s hands using the vector coherence mapping algorithm. We code the gestures to identify the catchments. We recover discourse purposes utilizing a system of guided questions. Finally, we define prosody in terms of the ToBI system. The results of these analyses are compared against the computed motion traces to identify the cues accessible in the gestural and audio data that correlate well with the psycholinguistic analyses. The results show that motion, prosody and discourse structure are integrated at each moment of speaking. The electronic edition of this article includes audio-visual data.

No abstract

No abstract

No abstract

Finger-tapping experiments were conducted to examine whether the dynamics of intrapersonal and interpersonal coordination systems can be described equally by the Haken—Kelso—Bunz model, which describes inter-limb coordination dynamics. This article reports the results of finger-tapping experiments conducted in both systems. Two within-subject factors were investigated: the phase mode and the number of fingers. In the intrapersonal experiment (Experiment 1), the participants were asked to tap, paced by a gradually hastening auditory metronome, looking at their fingers moving, using the index finger in the two finger condition, or the index and middle finger in the four-finger condition. In the interpersonal experiment (Experiment 2), pairs of participants performed the task while each participant used the outside hand, tapping with the index finger in the two finger condition, or the index and middle finger in the four-finger condition. Some results did not agree with the HKB model predictions. First, from Experiment 1, no significant difference was observed in the movement stability between the in-phase and anti-phase modes in the two finger condition. Second, from Experiment 2, no significant difference was found in the movement stability between the in-phase and anti-phase mode in the four-finger condition. From these findings, different coordination dynamics were inferred between intrapersonal and interpersonal coordination systems against prediction from the previous studies. Results were discussed according to differences between intrapersonal and interpersonal coordination systems in the availability of perceptual information and the complexity in the interaction between limbs derived from a nested structure.