Creating and Exploiting Multimodal Annotated Corpora: The ToMA Project

Blache, Philippe; Bertrand, Roxane; Ferré, Gaëlle

doi:10.1007/978-3-642-04793-0_3

Cited by 19 publications

(13 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We then drew up a list of occurrences of similar events in our dataset, i.e., a collection of examples, as recommended in CA (Mondada, 2013). However, unlike a CA -style transcription that exhibits all the relevant cues (prosodic, gestural, phonetic) on the same line (see (Selting et al, 1998) for the GAT system, adapted from the Jefferson-style transcription system), the transcription used in this work is one of the levels of the annotation process elaborated within the framework of the OTIM project (Blache et al, 2009). Using Praat (Boersma and Weenink, 2009), rich and systematic annotations related to the different linguistic domains, from phonetic to gestural, are provided.…”

Section: Methodsmentioning

confidence: 99%

Interactional convergence in conversational storytelling: when reported speech is a cue of alignment and/or affiliation

Guardiola

Bertrand

2013

Front. Psychol.

Self Cite

View full text Add to dashboard Cite

This paper investigates how and when interactional convergence is established by participants in conversation. We analyze sequences of storytelling using an original method that combines Conversation Analysis and a corpus-based approach. In storytelling, the participant in the position of “listener” is expected to produce either generic or specific responses adapted to the storyteller's narrative. The listener's behavior produced within the current activity is a cue of his/her interactional alignment. We show here that the listener can produce a specific type of (aligned) response, which we term a reported speech utterance in echo. The participant who is not telling the story is nonetheless able to animate the characters, while reversing the usual asymmetric roles of storyteller and listener. The use of this device is a way for the listener to display his/her stance toward the events told by the storyteller. If the listener's stance is congruent with that of the storyteller, this reveals a high degree of affiliation between the participants. We present seventeen excerpts from a collection of 94 instances of Echo Reported Speech (ERS) which we examined using the concepts of alignment and affiliation in order to show how different kinds of convergent sequences are constructed. We demonstrate that this phenomenon is mainly used by the listener to align and affiliate with the storyteller by means of reformulative, enumerative, or overbidding ERS. We also show that in affiliative sequences, reported speech can be used by the listener in a humorous way in order to temporarily disalign. This disalignment constitutes a potential starting point for an oblique sequence, which, if accepted and continued by the storyteller, gives rise to a highly convergent sequence.

show abstract

Section: Methodsmentioning

confidence: 99%

Interactional convergence in conversational storytelling: when reported speech is a cue of alignment and/or affiliation

Guardiola

Bertrand

2013

Front. Psychol.

Self Cite

View full text Add to dashboard Cite

show abstract

“…Unfortunately, annotating multimodal data is still problematic for both theoretical and technical reasons: first, there is a lack of linguistic and cognitive theories taking into consideration all the different aspects of multimodality; second, we need to specify a standardized way of representing multimodal information in order to give access to large multimodal corpora, as richly annotated as possible (Blache et al 2009). So there is still a long way to go in this field.…”

Section: Multimodal Corporamentioning

confidence: 99%

Multimodal Definition: The Multiplication of Meaning in Electronic Dictionaries

Liu

2015

LEXI

View full text Add to dashboard Cite

With the popularity of electronic dictionaries, multimodality plays an increasingly important role in lexicography. To broaden the horizons of e-dictionary definitions, this article argues for the establishment of a new term, multimodal definition, as a key component of multimodal lexicography. This term integrates verbal definitions and the complementary multimodal resources from a holistic viewpoint of meaning. In a multimodal definition, a dynamic meaning ecology can be formed, with two critical variables functioning, (semiotic) mode selection and intermodal synergy. In this ecology, meaning expressed verbally can be multiplied in four dimensions: content, form, space and time. Future directions for research are discussed, including dictionary user studies and multimodal corpora. The findings of this article shed light on the construction of a theoretical model for e-lexicography which remains an urgent task in the ongoing digital revolution.

show abstract

“…Because of this ambivalence, the interlocutors have to be very close to be able to perceive and/or 8 hours of audio-video recorded dialogues. The interactions are transcribed and annotated to be useful for different fields: phonetics, prosody, syntax, discourse and non-verbal analysis (see the ToMA project (OTIM in French), Blache et al 2009). This goal required a particular protocol of experiment, which allowed us to obtain dialogues interesting both for the interactional level and the phonetic and prosodic approach.…”

Section: Conversational Humormentioning

confidence: 99%

Does prosody play a specific role in conversational humor?

Bertrand

Priego-Valverde²

2011

P&C

Self Cite

View full text Add to dashboard Cite

In this paper we use Conversation Analysis (CA) to investigate conversational humor in talk-in-interaction. We attempt to better understand how the latter is produced and co-constructed by participants in accounting for the devices used by participants in their sequential environment. The framework of CA enables us to take into account the various means available to speakers to communicate, orient to the others, etc. From our data, reported speech, confirmation request/ answer, and repetitions appear as the main discursive devices to create humor (presented as the result of the appearance of the incongruity). We focused on prosodic cues that are strongly involved in these devices and then contribute to the humorous tonality of the talk. Finally, the co-construction of conversational humor is mainly described through the notion of orientation and prosodic orientation.

show abstract

Creating and Exploiting Multimodal Annotated Corpora: The ToMA Project

Cited by 19 publications

References 15 publications

Interactional convergence in conversational storytelling: when reported speech is a cue of alignment and/or affiliation

Interactional convergence in conversational storytelling: when reported speech is a cue of alignment and/or affiliation

Multimodal Definition: The Multiplication of Meaning in Electronic Dictionaries

Does prosody play a specific role in conversational humor?

Contact Info

Product

Resources

About