Incorporation of the ASR output in speaker segmentation and clustering within the task of speaker diarization of broadcast streams

Silovský, Jan; Žďánský, Jindřich; Nouza, Jan; Červa, Petr; Prażak, Jan

doi:10.1109/mmsp.2012.6343426

Cited by 15 publications

(11 citation statements)

References 7 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Standard metrics, such as the Word Error Rate (WER) and the Diarization Error Rate (DER) used in ASR and in diarization, respectively, are useful during modeling in order to have benchmarks and quantifiable areas of improvement. However, they do not necessarily reflect the transcript quality from a user's perspective (Silovsky, Zdansky, Nouza, Cerva, & Prazak, 2012) and they are not always representative of the performance with respect to semantics and to clinical impact (Miner et al, 2020). Qualitative surveys where experts share their opinions on the accuracy of the system output could assist highlighting specific areas of clinical importance on which the modeling efforts should focus.…”

Section: Metricmentioning

confidence: 99%

Automated Evaluation Of Psychotherapy Skills Using Speech And Language Technologies

Flemotomos,

Martinez,

Chen

et al. 2021

Preprint

View full text Add to dashboard Cite

With the growing prevalence of psychological interventions, it is vital to have measures which rate the effectiveness of psychological care, in order to assist in training, supervision, and quality assurance of services. Traditionally, quality assessment is addressed by human raters who evaluate recorded sessions along specific dimensions, often codified through constructs relevant to the approach and domain. This is however a cost-prohibitive and time-consuming method which leads to poor feasibility and limited use in real-world settings. To facilitate this process, we have developed an automated competency rating tool able to process the raw recorded audio of a session, analyzing who spoke when, what they said, and how the health professional used language to provide therapy. Focusing on a use case of a specific type of psychotherapy called Motivational Interviewing, our system gives comprehensive feedback to the therapist, including information about the dynamics of the session (e.g., therapist's vs. client's talking time), low-level psychological language descriptors (e.g., type of questions asked), as well as other high-level behavioral constructs (e.g., the extent to which the therapist understands the clients' perspective). We describe our platform and its performance, using a dataset of more than 5,000 recordings drawn from its deployment in a real-world clinical setting used to assist training of new therapists. We are confident that a widespread use of automated psychotherapy rating tools in the near future will augment experts' capabilities by providing an avenue for more effective training and skill improvement and will eventually lead to more positive clinical outcomes.

show abstract

Section: Metricmentioning

confidence: 99%

Automated Evaluation Of Psychotherapy Skills Using Speech And Language Technologies

Flemotomos,

Martinez,

Chen

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…The LVCSR system used for the transcription of archive documents employs a two-pass strategy. The output of the first decoder pass is used for a) segmentation to speech and non-speech parts, b) synchronization of speaker change detector with word and noise boundaries [10], c) speaker clustering [11], and d) speaker adaptation via the CMLLR technique [12]. The first pass is usually performed with a smaller lexicon to reduce computational costs and time.…”

Section: Speech Transcription Systemmentioning

confidence: 99%

“…A special label (COM ) is assigned to those Czech and Slovak words that share the same orthography and pronunciation. For each speech segment (determined by the speaker change point detector [10]), we get the numbers of recognized Czech words (N CZ ), Slovak words (N SK ) and the common ones (N COM ).The utterance in the segment is identified as Czech or Slovak according to the higher of counts N CZ and N SK .…”

Section: Lvcsr With Merged Lexicons and Language Modelsmentioning

confidence: 99%

Dealing with Bilingualism in Automatic Transcription of Historical Archive of Czech Radio

Nouza

Červa

Silovský

2013

New Trends in Image Analysis and Processing – ICIAP 2013

Self Cite

View full text Add to dashboard Cite

One of the biggest challenges in the automatic transcription of the historical audio archive of Czech and Czechoslovak radio is bilingualism. Two closely related languages, Czech and Slovak, are mixed in many archive documents. Both were the official languages in former Czechoslovakia (1918-1992) and both were used in media. The two languages are considered similar, although they differ in more than 75 % of their lexical inventories, which complicates automatic speech-to-text conversion. In this paper, we present and objectively measure the difference between the two languages. After that we propose a method suitable for automatic identification of two acoustically and lexically similar languages. It is based on employing 2 size-optimized parallel lexicons and language models. On large test data, we show that the 2 languages can be distinguished with almost 99 % accuracy. Moreover, the language identification module can be easily incorporated into a 2-pass decoding scheme with almost negligible additional computation costs. The proposed method has been employed in the project aimed at the disclosure of Czech and Czechoslovak oral cultural heritage.

show abstract

“…For example, the transcription of the input audio provides a strong clue to estimate the utterance boundaries. Several works were proposed to combine the automatic speech recognition (ASR) with speaker diarization, such as using the word boundary information from ASR [18,19] or improving the speaker segmentation and clustering based on the information from ASR [20,21]. While these works showed promising results, the ASR and speaker diarization models were separately trained.…”

Section: Introductionmentioning

confidence: 99%

Transcribe-to-Diarize: Neural Speaker Diarization for Unlimited Number of Speakers using End-to-End Speaker-Attributed ASR

Kanda¹,

Xiong²,

Gaur³

et al. 2021

Preprint

View full text Add to dashboard Cite

This paper presents Transcribe-to-Diarize, a new approach for neural speaker diarization that uses an end-to-end (E2E) speaker-attributed automatic speech recognition (SA-ASR). The E2E SA-ASR is a joint model that was recently proposed for speaker counting, multi-talker speech recognition, and speaker identification from monaural audio that contains overlapping speech. Although the E2E SA-ASR model originally does not estimate any time-related information, we show that the start and end times of each word can be estimated with sufficient accuracy from the internal state of the E2E SA-ASR by adding a small number of learnable parameters. Similar to the target-speaker voice activity detection (TS-VAD)-based diarization method, the E2E SA-ASR model is applied to estimate speech activity of each speaker while it has the advantages of (i) handling unlimited number of speakers, (ii) leveraging linguistic information for speaker diarization, and (iii) simultaneously generating speakerattributed transcriptions. Experimental results on the LibriCSS and AMI corpora show that the proposed method achieves significantly better diarization error rate than various existing speaker diarization methods when the number of speakers is unknown, and achieves a comparable performance to TS-VAD when the number of speakers is given in advance. The proposed method simultaneously generates speaker-attributed transcription with state-of-the-art accuracy.

show abstract

Incorporation of the ASR output in speaker segmentation and clustering within the task of speaker diarization of broadcast streams

Cited by 15 publications

References 7 publications

Automated Evaluation Of Psychotherapy Skills Using Speech And Language Technologies

Automated Evaluation Of Psychotherapy Skills Using Speech And Language Technologies

Dealing with Bilingualism in Automatic Transcription of Historical Archive of Czech Radio

Transcribe-to-Diarize: Neural Speaker Diarization for Unlimited Number of Speakers using End-to-End Speaker-Attributed ASR

Contact Info

Product

Resources

About