MCE 2018: The 1st Multi-Target Speaker Detection and Identification Challenge Evaluation

Abstract

The Multi-target Challenge 1 aims to assess how well current speech technology is able to determine whether or not a recorded utterance was spoken by one of a large number of blacklisted speakers. It is a form of multi-target speaker detection based on real-world telephone conversations. Data recordings are generated from call center customer-agent conversations. The task is to measure how accurately one can detect 1) whether a test recording is spoken by a blacklisted speaker, and 2) which specific blacklisted speaker was talking. This paper outlines the challenge and provides its baselines, results, and discussions.

Latent Space Representation for Multi-Target Speaker Detection and Identification with a Sparse Dataset Using Triplet Neural Networks

“…The MCE 2018 challenge provides a dataset that consists of recordings of conversations of both blacklisted and nonblacklisted speakers. These recordings are obtained from a call center, whereby some of the customers have been marked as blacklisted [15]. The exact reasons for blacklisting customers are not given, only the label (blacklisted and nonblacklisted) is provided for each recording.…”

“…In this paper, our main focus is on speaker identification, which we test using the dataset from the Multi-Target Speaker Detection and Identification Challenge [14], a competition focused on the task of automatic blacklisted speaker detection using audio recordings. This dataset 1 consists of i-vectors [15] of length 600. Each of these i-vectors corresponds to a real-world telephone conversation by customers and agents from a call-center.…”

“…Much of the existing research on multi-target speaker detection uses larger datasets than the aforementioned to train their models, for example in [11,10], they have 300 utterances for each speaker; other work by Li et al [12] uses 10-50 utterances for each speaker. The MCE 2018 dataset [15], however, is challenging as there are only three samples for each speaker.…”

“…Despite the successful results achieved by the above mentioned methods, neural network approaches are becoming ever more popular. We explore the power of TNNs on classification with a small dataset, i.e., MCE 2018 [15].…”

“…Task 2 is a speaker identification task whereby each of the 3,631 unique blacklisted speakers form a separate class (i.e., to identify which specific blacklisted speaker was talking). The MCE 2018 [15] dataset does not provide the original audio files, but only the calculated i-vectors. We will therefore use these as input to the TNN model.…”