Introducing the 3MT_French Dataset to Investigate the Timing of Public Speaking Judgements

Abstract

In most public speaking datasets, judgements are given after watching the entire performance, or on thin slices randomly selected from the presentations, without focusing on the temporal location of these slices. This does not allow to investigate how people's judgements develop over time during presentations. This contrasts with primacy and recency theories, which suggest that some moments of the speech could be more salient than others and contribute disproportionately to the perception of the speaker's performance.To provide novel insights on this phenomenon, we present the 3MT_French dataset. It contains a set of public speaking annotations collected on a crowd-sourcing platform through a novel annotation scheme and protocol. Global evaluation, persuasiveness, perceived self-confidence of the speaker and audience engagement were annotated on different time windows (i.e., the beginning, middle or end of the presentation, or the full video). This new resource will be useful to researchers working on public speaking assessment and training. It will allow to fine-tune the analysis of presentations under a novel perspective relying on socio-cognitive theories rarely studied before in this context, such as first impressions and primacy and recency theories. An exploratory correlation analysis on the annotations provided in the dataset suggests that the early moments of a presentation have a stronger impact on the judgements.

Insights Into the Importance of Linguistic Textual Features on the Persuasiveness of Public Speaking

“…We leveraged the 3MT_French dataset [Biancardi et al 2022], which consists of annotated 3-minute video recordings of presentations in the French scientific public speaking competition "Ma Thèse en 180 2 https://github.com/anonympapers/textual_features_importance.git seconds". This dataset includes presentations from both female (135 speakers) and male (113 speakers) participants, covering diverse topics and showcasing the thesis works of French PhD students.…”

“…Each video was evaluated by three different viewers through the Amazon Mechanical Turk [Mason and Suri 2011] crowd-sourcing platform. To mitigate the impact of low inter-rater agreement (measured by intraclass correlation coefficient (ICC) [Bartko 1966]), we applied the root mean square (RMS) to the ratings provided by the three annotators as the final scoring method, similar to the approach followed in [Dinkar et al 2020a] (for more details, refer to the paper [Biancardi et al 2022]). In order to analyse the speech transcripts, we processed the data set using a speech transcription library 3 .…”