Unsupervised neural network based feature extraction using weak top-down constraints

Kamper, Herman; Elsner, Micha; Jansen, Aren; Goldwater, Sharon

doi:10.1109/icassp.2015.7179087

Cited by 100 publications

(134 citation statements)

References 20 publications

Supporting

Mentioning

134

Contrasting

Order By: Relevance

“…Other methods incorporate weak top-down supervision by first extracting pairs of similar word-or phraselike units using unsupervised term detection, and using these to constrain the representation learning. Examples include the correspondence autoencoder (cAE) [3] and ABNet [13]. Both aim to learn representations that make similar pairs even more similar; the ABNet additionally tries to make different pairs more different.…”

Section: A Background and Motivationmentioning

confidence: 99%

Multilingual and unsupervised subword modeling for zero-resource languages

Hermann

Kamper

Goldwater

2021

Computer Speech & Language

Self Cite

View full text Add to dashboard Cite

Unsupervised subword modeling aims to learn lowlevel representations of speech audio in "zero-resource" settings: that is, without using transcriptions or other resources from the target language (such as text corpora or pronunciation dictionaries). A good representation should capture phonetic content and abstract away from other types of variability, such as speaker differences and channel noise. Previous work in this area has primarily focused on learning from target language data only, and has been evaluated only intrinsically. Here we directly compare multiple methods, including some that use only target language speech data and some that use transcribed speech from other (non-target) languages, and we evaluate using two intrinsic measures as well as on a downstream unsupervised word segmentation and clustering task. We find that combining two existing target-language-only methods yields better features than either method alone. Nevertheless, even better results are obtained by extracting target language bottleneck features using a model trained on other languages. Cross-lingual training using just one other language is enough to provide this benefit, but multilingual training helps even more. In addition to these results, which hold across both intrinsic measures and the extrinsic task, we discuss the qualitative differences between the different types of learned features.

show abstract

Section: A Background and Motivationmentioning

confidence: 99%

Multilingual and unsupervised subword modeling for zero-resource languages

Hermann

Kamper

Goldwater

2021

Computer Speech & Language

Self Cite

View full text Add to dashboard Cite

show abstract

“…To obtain useful features, it is essential to pretrain the CAE as a conventional AE [31]. Our CAE has the same structure as the AE described in Section 5.1 and pretraining follows the same procedure described there.…”

Section: Correspondence Autoencoder Featuresmentioning

confidence: 99%

Feature Exploration for Almost Zero-Resource ASR-Free Keyword Spotting Using a Multilingual Bottleneck Extractor and Correspondence Autoencoders

et al. 2019

Self Cite

View full text Add to dashboard Cite

We compare features for dynamic time warping (DTW) when used to bootstrap keyword spotting (KWS) in an almost zeroresource setting. Such quickly-deployable systems aim to support United Nations (UN) humanitarian relief efforts in parts of Africa with severely under-resourced languages. Our objective is to identify acoustic features that provide acceptable KWS performance in such environments. As supervised resource, we restrict ourselves to a small, easily acquired and independently compiled set of isolated keywords. For feature extraction, a multilingual bottleneck feature (BNF) extractor, trained on well-resourced out-of-domain languages, is integrated with a correspondence autoencoder (CAE) trained on extremely sparse in-domain data. On their own, BNFs and CAE features are shown to achieve a more than 2% absolute performance improvement over baseline MFCCs. However, by using BNFs as input to the CAE, even better performance is achieved, with a more than 11% absolute improvement in ROC AUC over MFCCs and more than twice as many top-10 retrievals for two evaluated languages, English and Luganda. We conclude that integrating BNFs with the CAE allows both large out-of-domain and sparse in-domain resources to be exploited for improved ASR-free keyword spotting.

show abstract

“…Another type of neural approach that has seen success in learning acoustic features is the correspondence autoencoder (CAE) [28]. Instead of trying to reconstruct its input, as is done in a standard autoencoder [29], the CAE tries to reconstruct another instance of the same type as its input.…”

Section: Introductionmentioning

confidence: 99%

Unsupervised Feature Learning for Speech Using Correspondence and Siamese Networks

Engelbrecht

Kamper

2020

IEEE Signal Process. Lett.

Self Cite

View full text Add to dashboard Cite

In zero-resource settings where transcribed speech audio is unavailable, unsupervised feature learning is essential for downstream speech processing tasks. Here we compare two recent methods for frame-level acoustic feature learning. For both methods, unsupervised term discovery is used to find pairs of word examples of the same unknown type. Dynamic programming is then used to align the feature frames between each word pair, serving as weak top-down supervision for the two models. For the correspondence autoencoder (CAE), matching frames are presented as input-output pairs. The Triamese network uses a contrastive loss to reduce the distance between frames of the same predicted word type while increasing the distance between negative examples. For the first time, these feature extractors are compared on the same discrimination tasks using the same weak supervision pairs. We find that, on the two datasets considered here, the CAE outperforms the Triamese network. However, we show that a new hybrid correspondence-Triamese approach (CTriamese), consistently outperforms both the CAE and Triamese models in terms of average precision and ABX error rates on both English and Xitsonga evaluation data.

show abstract

Unsupervised neural network based feature extraction using weak top-down constraints

Cited by 100 publications

References 20 publications

Multilingual and unsupervised subword modeling for zero-resource languages

Multilingual and unsupervised subword modeling for zero-resource languages

Feature Exploration for Almost Zero-Resource ASR-Free Keyword Spotting Using a Multilingual Bottleneck Extractor and Correspondence Autoencoders

Unsupervised Feature Learning for Speech Using Correspondence and Siamese Networks

Contact Info

Product

Resources

About