No abstract

The present research explores non-verbal behavior that accompanies the management of turns in naturally occurring conversations. To analyze turn management, we implemented the ISO 24617-2 multidimensional dialog act annotation scheme. The classification of the communicative intent of non-verbal behavior was performed with the annotation scheme for spontaneous authentic communication called the EVA annotation scheme. Both dialog acts and non-verbal communicative intent were observed according to their underlying nature and information exchange channel. Both concepts were divided into foreground and background expressions. We hypothesize that turn management dialog acts, being a background expression, co-occur with communication regulators, a class of non-verbal communicative intent, which are also of background nature. Our case analysis confirms this hypothesis. Furthermore, it reveals that another group of non-verbal communicative intent, the deictics, also often accompany turn management dialog acts. As deictics can be both foreground and background expressions, the premise that background non-verbal communicative intent is interlinked with background dialog acts is upheld. And when deictics were perceived as part of the foreground they co-occurred with foreground dialog acts. Therefore, dialog acts and non-verbal communicative intent share the same underlying nature, which implies a duality of the two concepts.

Translation and interpreting are not only linguistic activities, but also to a large extent primarily activities of cultural transfer. Transcultural communication displays complexity, diversity and readiness for conflict in communicative interaction, so the interpreter/translator, as an intercultural mediator, is assigned a special communicative role in this regard. This article examines how interpreters at the European Parliament deal with controversial language rendering evaluative components of political statements as well as whether there is a rise in stress-related disfluencies in the interpretation of such statements and whether intonation (dis)similarities between the source text and the interpretations occur in the context of cultural and lexical know-how. Seven excerpts from four sessions of the European Parliament in the last six years and their interpretations into Croatian, Slovene, English, French and German were analysed from the point of view of stress and culture. Deviations in pitch and intensity levels of both the speaker and the interpreters were calculated and statistically compared in the light of differing cultural know-how. The intonation results for these interpreting examples showed that all the interpreters followed the speaker’s pitch deviations to a certain extent. Analysis of politically-controversial statements also revealed that more than 80% of the interpretations selected contained stress-related disfluencies and almost 70% contained some form of discrepancy with the source text at a lexical level. The interpretations therefore largely contained fewer negative evaluative components of controversial language than the speakers in the European Parliament.

This paper focuses on gaining new knowledge through observation, qualitative analytics, and cross-modal fusion of rich multi-layered conversational features expressed during multiparty discourse. The outlined research stems from the theory that speech and co-speech gestures originate from the same representation; however, the representation is not solely limited to the speech production process. Thus, the nature of how information is conveyed by synchronously fusing speech and gestures must be investigated in detail. Therefore, this paper introduces an integrated annotation scheme and methodology which opens the opportunity to study verbal (i.e., speech) and non-verbal (i.e., visual cues with a communicative intent) components independently, however, still interconnected over a common timeline. To analyse this interaction between linguistic, paralinguistic, and non-verbal components in multiparty discourse and to help improve natural language generation in embodied conversational agents, a high-quality multimodal corpus, consisting of several annotation layers spanning syntax, POS, dialogue acts, discourse markers, sentiment, emotions, non-verbal behaviour, and gesture units was built and is represented in detail. It is the first of its kind for the Slovenian language. Moreover, detailed case studies show the tendency of metadiscourse to coincide with non-verbal behaviour of non-propositional origin. The case analysis further highlights how the newly created conversational model and the corresponding information-rich consistent corpus can be exploited to deepen the understanding of multiparty discourse.

The research proposed in this paper focuses on pragmatic interlinks between discourse markers and non-verbal behavior. Although non-verbal behavior is recognized to add non-redundant information and social interaction is not merely recognized as the transmission of words and sentences, the evidence regarding grammatical/linguistic interlinks between verbal and non-verbal concepts are vague and limited to restricted domains. This is even more evident when non-verbal behavior acts in the foreground but contributes to the structure and organization of the discourse. This research focuses on investigating the multimodal nature of discourse markers by observing their linguistic and paralinguistic properties in informal discourse. We perform a quantitative analysis with case studies for representative cases. The results show that discourse markers and background non-verbal behavior tend to follow a similar functionality in interaction. Therefore, by examining them together, one gains more insight into their true intent despite the high multifunctionality of both non-verbal behavior and DMs.