Analyzing Speaker Information in Self-Supervised Models to Improve Zero-Resource Speech Processing

Niekerk, Benjamin van; Nortje, Leanne; Kamper, Herman

doi:10.21437/interspeech.2021-1182

Cited by 20 publications

(12 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Selfsupervised speech models can produce continuous features that accurately capture phonetic contrasts while being invariant to nuisance factors (e.g. speaker) [12], [28]- [30]. To obtain discrete units, a self-supervised model can be coupled with a vector quantization (VQ) module, either as part of the model itself or by introducing a clustering step after training [29]- [34].…”

Section: Dpdp For Acoustic Unit Discoverymentioning

confidence: 99%

Word Segmentation on Discovered Phone Units with Dynamic Programming and Self-Supervised Scoring

Kamper¹

2022

Preprint

Self Cite

View full text Add to dashboard Cite

Recent work on unsupervised speech segmentation has used self-supervised models with a phone segmentation module and a word segmentation module that are trained jointly. This paper compares this joint methodology with an older idea: bottom-up phone-like unit discovery is performed first, and symbolic word segmentation is then performed on top of the discovered units (without influencing the lower level). I specifically describe a duration-penalized dynamic programming (DPDP) procedure that can be used for either phone or word segmentation by changing the self-supervised scoring network that gives segment costs. For phone discovery, DPDP is applied with a contrastive predictive coding clustering model, while for word segmentation it is used with an autoencoding recurrent neural network. The two models are chained in order to segment speech. This approach gives comparable word segmentation results to state-of-the-art joint self-supervised models on an English benchmark. On French and Mandarin data, it outperforms previous systems on the ZeroSpeech benchmarks. Analysis shows that the chained DPDP system segments shorter filler words well, but longer words might require an external top-down signal.

show abstract

Section: Dpdp For Acoustic Unit Discoverymentioning

confidence: 99%

Word Segmentation on Discovered Phone Units with Dynamic Programming and Self-Supervised Scoring

Kamper¹

2022

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…System BN [25] begins with the baseline CPC representations and applies speaker normalization before re-running kmeans. An LSTM language model architecture is used.…”

Section: Submitted Systems and Resultsmentioning

confidence: 99%

The Zero Resource Speech Challenge 2021: Spoken language modelling

Dunbar¹,

Bernard²,

Hamilakis³

et al. 2021

Preprint

View full text Add to dashboard Cite

We present the Zero Resource Speech Challenge 2021, which asks participants to learn a language model directly from audio, without any text or labels. The challenge is based on the Librilight dataset, which provides up to 60k hours of audio from English audio books without any associated text. We provide a pipeline baseline system consisting on an encoder based on contrastive predictive coding (CPC), a quantizer (k-means) and a standard language model (BERT or LSTM). The metrics evaluate the learned representations at the acoustic (ABX discrimination), lexical (spot-the-word), syntactic (acceptability judgment) and semantic levels (similarity judgment). We present an overview of the eight submitted systems from four groups and discuss the main results.

show abstract

“…Then, an SSRL algorithm called contrastive predictive coding (CPC) [17] is applied to the data. CPC aims to produce linearly separable features that can be used to predict signal evolution over time [20,21]. CPC has already been successfully used with clustering-based approaches (e.g.…”

Section: Methodsmentioning

confidence: 99%

“…CPC has already been successfully used with clustering-based approaches (e.g. [21][22][23]) and also produces features that separate suprasegmental properties such as speaker identities [17]. However, to the best of our knowledge, [24] is the only study so far using CPC for AL.…”

Section: Methodsmentioning

confidence: 99%

Analysis of Self-Supervised Learning and Dimensionality Reduction Methods in Clustering-Based Active Learning for Speech Emotion Recognition

Vaaras¹,

Airaksinen²,

Räsänen³

2022

Preprint

View full text Add to dashboard Cite

When domain experts are needed to perform data annotation for complex machine-learning tasks, reducing annotation effort is crucial in order to cut down time and expenses. For cases when there are no annotations available, one approach is to utilize the structure of the feature space for clustering-based active learning (AL) methods. However, these methods are heavily dependent on how the samples are organized in the feature space and what distance metric is used. Unsupervised methods such as contrastive predictive coding (CPC) can potentially be used to learn organized feature spaces, but these methods typically create highdimensional features which might be challenging for estimating data density. In this paper, we combine CPC and multiple dimensionality reduction methods in search of functioning practices for clustering-based AL. Our experiments for simulating speech emotion recognition system deployment show that both the local and global topology of the feature space can be successfully used for AL, and that CPC can be used to improve clustering-based AL performance over traditional signal features. Additionally, we observe that compressing data dimensionality does not harm AL performance substantially, and that 2-D feature representations achieved similar AL performance as higher-dimensional representations when the number of annotations is not very low.

show abstract

Analyzing Speaker Information in Self-Supervised Models to Improve Zero-Resource Speech Processing

Cited by 20 publications

References 0 publications

Word Segmentation on Discovered Phone Units with Dynamic Programming and Self-Supervised Scoring

Word Segmentation on Discovered Phone Units with Dynamic Programming and Self-Supervised Scoring

The Zero Resource Speech Challenge 2021: Spoken language modelling

Analysis of Self-Supervised Learning and Dimensionality Reduction Methods in Clustering-Based Active Learning for Speech Emotion Recognition

Contact Info

Product

Resources

About