Speaker role recognition to help spontaneous conversational speech detection

Multimodal embedding fusion for robust speaker role recognition in video broadcast

“…In the literature, SRR methods have been studied in purpose of chaptering audio-visual documents (talk shows and broadcast news). Existing methods are divided according to the features extracted (audio and/or text), the decision level (for each speaker turn [2,3] or globally on all the turns of a given speaker [4,5,6,7,8]) and classification techniques (supervised [4,5,2,3] or unsupervised [6,7,8]).…”

“…A loss of 1.1% in accuracy is shown when automatic features are extracted compared to the use of manually labelled linguistic phenomena. In [5], authors used temporal, acoustic and prosodic features to classify roles at the speaker cluster level. Authors distinguish between punctual and non punctual speaker roles and train Support Vector Machine (SVM) and Gaussian Mixture Model (GMM) classifiers hierarchically.…”

Combined Speaker Clustering and Role Recognition in Conversational Speech

“…Speaker Role Recognition (SRR) is the task of assigning a specific role to each speaker turn (speaker-homogeneous segment) in a speech signal. This task plays a significant role in numerous areas, such as information retrieval [1], audio indexing [2], or social interaction analysis [3]. Most of the research efforts have been focused on identifying roles in broadcast news programs or talk shows [4][5][6][7] , while there have been also works dealing with meeting scenarios [8], conferences [9], medical discussions between domain experts [10], and psychotherapy sessions [11].…”

“…In the case of speaker-level SRR (Figure 1b), the classifier is built in two steps, the first being a Speaker Clustering (SC) algorithm, or a diarization system in the more general case, where the turns are grouped into same-speaker clusters in an unsupervised way and then each cluster is assigned a specific role. In this line of work, [16] uses a social network analysis approach taking into consideration relational data across different speakers, while a hierarchical classification system is proposed in [2] and [12]. The effect of various modalities on the final performance of SRR when using boosting algorithms is investigated in [17].…”

Boosting bonsai trees for efficient features combination: application to speaker role identification

“…In the literature, the problem is seen as a multiclass classification problem where each speaker of a show has to be associated with a role label. In this way, some previous studies have tackled the problem using machine learning from mainly lexical features extracted from the transcription [1,2,3], from acoustic / prosodic features [4,5], or from a combination of lexical and acoustic features [6]. Those studies have highlighted the efficiency of a boosting algorithm over decision stumps to combine the various features.…”