A Multimodal Model for Predicting Feedback Position and Type During Conversation

Boudin, Auriane; Bertrand, Roxane; Rauzy, Stéphane; Ochs, Magalie; Blache, Philippe

doi:10.2139/ssrn.4143334

Cited by 2 publications

(1 citation statement)

References 59 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The smile interval labels and boundaries are predicted from the intensities of the facial Action Units outputted by OpenFace. For the present study the SMAD output is finally transformed in a 3 level smiles scale (NF, LI, HI ) with NF encoding Neutral Face, LI Low Intensity smiles (smiles with mouth closed) and HI High Intensity smiles (smiles with mouth opened), as proposed in [14].…”

Section: Automatic Annotationmentioning

confidence: 99%

SMYLE: A new multimodal resource of talk-in-interaction including neuro-physiological signal

Boudin,

Bertrand,

Rauzy

et al. 2023

International Cconference on Multimodal Interaction

View full text Add to dashboard Cite

This article presents the SMYLE corpus, the first multimodal corpus in French (16h) including neuro-physiological data from 60 participants engaged in face-to-face storytelling (8.2h) and free conversation tasks (7.8h). The originality of this corpus lies first in the fact that it bears all modalities, precisely synchronized and second in the addition for the first time at this scale of neuro-physiological modalities. It constitutes the first corpus of this size offering the opportunity to investigate cognitive characteristics of spontaneous conversation including at the brain level. The storytelling task comprises two conditions: a storyteller talking with a "normal" or a "distracted" listener. Contrasting normal and disrupted conversations allows to study at a behavioral, linguistic and cognitive levels the complex characteristics and organization of conversations.In this article, we present first the methodology developed to acquire and synchronize the different sources and types of signal. In a second part, we detail the large set of automatic, semi-automatic and manual annotations of the complete dataset. In a last section,

show abstract

Section: Automatic Annotationmentioning

confidence: 99%