Distinct kinematic markers of demonstration and joint action coordination? Evidence from virtual xylophone playing.

McEllin, Luke; Knoblich, Günther; Sebanz, Natalie

doi:10.1037/xhp0000505

Cited by 38 publications

(39 citation statements)

References 28 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…When compared to interacting with other adults, child-directed (Brand et al, 2002) as well as robot-directed actions (Vollmer et al, 2009) are modulated by distinct kinematic features. Similar features may also be useful in differentiating between various adult interactive contexts, such as demonstration and joint action coordination (McEllin, Knoblich, & Sebanz, 2018). Clinicians may also benefit from such analysis, as pantomime production is often used when assessing aphasia (Goldenberg, Hermsdörfer, Glindemann, Rorden, & Karnath, 2007; Hermsdörfer, Li, Randerath, Goldenberg, & Johannsen, 2012).…”

Section: Discussionmentioning

confidence: 99%

Toward the markerless and automatic analysis of kinematic features: A toolkit for gesture and movement research

et al. 2018

View full text Add to dashboard Cite

Action, gesture, and sign represent unique aspects of human communication that use form and movement to convey meaning. Researchers typically use manual coding of video data to characterize naturalistic, meaningful movements at various levels of description, but the availability of markerless motion-tracking technology allows for quantification of the kinematic features of gestures or any meaningful human movement. We present a novel protocol for extracting a set of kinematic features from movements recorded with Microsoft Kinect. Our protocol captures spatial and temporal features, such as height, velocity, submovements/strokes, and holds. This approach is based on studies of communicative actions and gestures and attempts to capture features that are consistently implicated as important kinematic aspects of communication. We provide opensource code for the protocol, a description of how the features are calculated, a validation of these features as quantified by our protocol versus manual coders, and a discussion of how the protocol can be applied. The protocol effectively quantifies kinematic features that are important in the production (e.g., characterizing different contexts) as well as the comprehension (e.g., used by addressees to understand intent and semantics) of manual acts. The protocol can also be integrated with qualitative analysis, allowing fast and objective demarcation of movement units, providing accurate coding even of complex movements. This can be useful to clinicians, as well as to researchers studying multimodal communication or human-robot interactions. By making this protocol available, we hope to provide a tool that can be applied to understanding meaningful movement characteristics in human communication. Keywords Communication . Methodology . Motion trackingHuman communication is intrinsically multimodal, consisting of not only speech but also visible communicative signals. Gesture, sign, and communicative actions (e.g., joint actions, demonstrations) are well-studied examples of communicative manual acts that can convey meaning in the presence or absence of co-occurring speech. A plethora of research in the last decade has shown that each of these modalities, although unique in certain ways, effectively utilizes movement and configuration to convey meaning and contribute to successful communication.Among an array of visual bodily cues that people resort to when conveying meaning, gestures stand out as a unique attribute of the human communication system. A wealth of research has shown that gestures (we use the term Bgesturesĥ ere to refer to movements of the hands and arms that are used to depict objects, ideas, events, and experiences; Kendon, 2004;McNeill, 1994) form an important aspect of communication. The study of gesture has opened a new window into human language,

show abstract

Section: Discussionmentioning

confidence: 99%

Toward the markerless and automatic analysis of kinematic features: A toolkit for gesture and movement research

et al. 2018

View full text Add to dashboard Cite

show abstract

“…An important goal for future research is to study which additional means experts use to provide a temporal scaffold for novices in teaching situations when novices also receive feedback from experts’ performance, and when experts and novices can also visually perceive each other’s actions. Some possible options include exaggerating movements ( McEllin, Knoblich, & Sebanz, 2017 ), reducing performance variability ( Vesper, van der Wel, Knoblich, & Sebanz, 2011 ), and relying on emerging leader-follower dynamics ( Konvalinka, Vuust, Roepstorff, & Frith, 2010 ). A further interesting question is whether and when teachers strategically avoid adjusting to novices to maximize learning opportunities for their students.…”

Section: Discussionmentioning

confidence: 99%

Joint action coordination in expert-novice pairs: Can experts predict novices’ suboptimal timing?

Wolf¹,

Sebanz²,

Knoblich³

2018

Cognition

Self Cite

View full text Add to dashboard Cite

Previous research has established that skilled joint action partners use predictive models to achieve temporal coordination, for instance, when playing a music duet. But how do joint action partners with different skill levels achieve coordination? Can experts predict the suboptimal timing of novices? What kind of information allows them to predict novices’ timing? To address these questions, we asked skilled pianists to perform duets with piano novices. We varied whether, prior to performing duets, experts were familiar with novices’ performances of their individual parts of the duets and whether experts had access to the musical scores including the novices’ part of the duet. Familiarity with the score led to better coordination when the score implied a difficult passage. Familiarity with novices’ performances led to better joint action coordination for the remaining parts of the duet. Together, the results indicate that experts are surprisingly flexible in predicting novices’ suboptimal timing.

show abstract

“…higher in space (Hilliard & Cook, 2016) when conveying novel information. Instrumental actions intended to teach show similar kinematic modulation, including spatial (McEllin, Knoblich, & Sebanz, 2018;Vesper & Richardson, 2014) and temporal (McEllin et al, 2018) exaggeration. Evidence from our own lab corroborates these findings of spatial and temporal modulation in the production of both actions and gestures.…”

Section: Introductionmentioning

confidence: 95%

The communicative advantage: how kinematic signaling supports semantic comprehension

Trujillo

Simanova

Bekkering

et al. 2019

Psychological Research

View full text Add to dashboard Cite

Humans are unique in their ability to communicate information through representational gestures which visually simulate an action (eg. moving hands as if opening a jar). Previous research indicates that the intention to communicate modulates the kinematics (e.g., velocity, size) of such gestures. If and how this modulation influences addressees' comprehension of gestures have not been investigated. Here we ask whether communicative kinematic modulation enhances semantic comprehension (i.e., identification) of gestures. We additionally investigate whether any comprehension advantage is due to enhanced early identification or late identification. Participants (n = 20) watched videos of representational gestures produced in a more-(n = 60) or less-communicative (n = 60) context and performed a forced-choice recognition task. We tested the isolated role of kinematics by removing visibility of actor's faces in Experiment I, and by reducing the stimuli to stick-light figures in Experiment II. Three video lengths were used to disentangle early identification from late identification. Accuracy and response time quantified main effects. Kinematic modulation was tested for correlations with task performance. We found higher gesture identification performance in more-compared to less-communicative gestures. However, early identification was only enhanced within a full visual context, while late identification occurred even when viewing isolated kinematics. Additionally, temporally segmented acts with more post-stroke holds were associated with higher accuracy. Our results demonstrate that communicative signaling, interacting with other visual cues, generally supports gesture identification, while kinematic modulation specifically enhances late identification in the absence of other cues. Results provide insights into mutual understanding processes as well as creating artificial communicative agents.

show abstract

Distinct kinematic markers of demonstration and joint action coordination? Evidence from virtual xylophone playing.

Cited by 38 publications

References 28 publications

Toward the markerless and automatic analysis of kinematic features: A toolkit for gesture and movement research

Toward the markerless and automatic analysis of kinematic features: A toolkit for gesture and movement research

Joint action coordination in expert-novice pairs: Can experts predict novices’ suboptimal timing?

The communicative advantage: how kinematic signaling supports semantic comprehension

Contact Info

Product

Resources

About