Information Retrieval for ZeroSpeech 2021: The Submission by University of Wroclaw

Chorowski, Jan; Ciesielski, Grzegorz; Dzikowski, Jarosław; Lancucki, Adrian; Marxer, Ricard; Opala, Mateusz; Pusz, Piotr; Rychlikowski, Paweł; Stypułkowski, Michał

doi:10.21437/interspeech.2021-1465

Cited by 10 publications

(2 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The challenge contains two tracks, the non-visuallygrounded track and the visually-grounded (VG) track. The non-VG track was introduced earlier, and submissions to this track studied the effect of filtering out speaker information (v. Niekerk et al 2021), denoising CPC (v. d. Oord, Li, andVinyals 2018) representations by using methods from information retrieval (Chorowski et al 2021), and combining CPC with deep clustering (Maekaku et al 2021). These ideas offered improved results on the phonetic, lexical and syntactic tasks.…”

Section: Related Workmentioning

confidence: 99%

Self-Supervised Representation Learning for Speech Using Visual Grounding and Masked Language Modeling

Puyuan¹,

Harwath²

2022

Preprint

View full text Add to dashboard Cite

In this paper, we describe our submissions to the ZeroSpeech 2021 Challenge and SUPERB benchmark. Our submissions are based on the recently proposed FaST-VGS model, which is a Transformer-based model that learns to associate raw speech waveforms with semantically related images, all without the use of any transcriptions of the speech. Additionally, we introduce a novel extension of this model, FaST-VGS+ 1 , which is learned in a multi-task fashion with a masked language modeling objective in addition to the visual grounding objective. On ZeroSpeech 2021, we show that our models perform competitively on the ABX task, outperform all other concurrent submissions on the Syntactic and Semantic tasks, and nearly match the best system on the Lexical task. On the SUPERB benchmark, we show that our models also achieve strong performance, in some cases even outperforming the popular wav2vec2.0 model.

show abstract

Section: Related Workmentioning

confidence: 99%

Self-Supervised Representation Learning for Speech Using Visual Grounding and Masked Language Modeling

Puyuan¹,

Harwath²

2022

Preprint

View full text Add to dashboard Cite

show abstract

“…al. [45] use the speaker information in the training. It is note worthy that the proposed HUC approach is low GPU budget (150h) and does not use any speaker information.…”

Section: E Comparison With Other Benchmarks On Abx Taskmentioning

confidence: 99%

Self Supervised Representation Learning with Deep Clustering for Acoustic Unit Discovery from Raw Speech

Krishna

Ganapathy

2022

ICASSP 2022 - 2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

View full text Add to dashboard Cite

The representation learning of speech, without textual resources, is an area of significant interest for many low resource speech applications. In this paper, we describe an approach to self-supervised representation learning from raw audio using a hidden unit clustering (HUC) framework. The input to the model consists of audio samples that are windowed and processed with 1-D convolutional layers. The learned "time-frequency" representations from the convolutional neural network (CNN) module are further processed with long short term memory (LSTM) layers which generate a contextual vector representation for every windowed segment. The HUC framework, allowing the categorization of the representations into a small number of phoneme-like units, is used to train the model for learning semantically rich speech representations. The targets consist of phoneme-like pseudo labels for each audio segment and these are generated with an iterative k-means algorithm. We explore techniques that improve the speaker invariance of the learned representations and illustrate the effectiveness of the proposed approach on two settings, i) completely unsupervised speech applications on the sub-tasks described as part of the ZeroSpeech 2021 challenge and ii) semi-supervised automatic speech recognition (ASR) applications on the TIMIT dataset and on the GramVaani challenge Hindi dataset. In these experiments, we achieve state-of-art results for various Zerospeech tasks. Further, on the ASR experiments, the HUC representations are shown to improve significantly over other established benchmarks based on wav2vec, HuBERT and Best-RQ.

show abstract

A Tool for Scholarly Phonemic Transcription of Ukrainian Texts

Vakulenko

2022

2022 12th International Conference on Advanced Computer Information Technologies (ACIT)

View full text Add to dashboard Cite

Information Retrieval for ZeroSpeech 2021: The Submission by University of Wroclaw

Cited by 10 publications

References 12 publications

Self-Supervised Representation Learning for Speech Using Visual Grounding and Masked Language Modeling

Self-Supervised Representation Learning for Speech Using Visual Grounding and Masked Language Modeling

Self Supervised Representation Learning with Deep Clustering for Acoustic Unit Discovery from Raw Speech

A Tool for Scholarly Phonemic Transcription of Ukrainian Texts

Contact Info

Product

Resources

About