An Attentional Model for Speech Translation Without Transcription

Abstract

For many low-resource languages, spoken language resources are more likely to be annotated with translations than transcriptions. This bilingual speech data can be used for word-spotting, spoken document retrieval, and even for documentation of endangered languages. We experiment with the neural, attentional model applied to this data. On phoneto-word alignment and translation reranking tasks, we achieve large improvements relative to several baselines. On the more challenging speech-to-word alignment task, our model nearly matches GIZA++'s performance on gold transcriptions, but without recourse to transcriptions or to a lexicon.

End-to-End Speech Translation with Knowledge Distillation

“…Anastasopoulos et al [6] use k-means clustering to cluster repeated audio patterns and automatically align spoken words with their translations. Duong et al [7] focus on the alignment between speech and translated phrase but not to directly predict the final translations. Bérard et al [8] give the first proof of the potential for end-to-end speech-to-text translation without using source language.…”

Unsupervised Word Segmentation from Speech with Attention

“…∀t, i αi,t = 1, with i indexing the source symbols). However, there is no similar constraint for the source symbols, as discussed by [5]. Rather than enforcing additional constraints on the alignments, as in the latter reference, we propose to reverse the architecture and to translate from WRL words into UL symbols, following [9].…”

“…This speech dataset was collected following a real language documentation scenario, using Lig Aikuma, 4 a mobile app specifically dedicated to fieldwork language documentation, which works both on Android powered smartphones and tablets [8]. The corpus is multilingual (5,130 Mboshi speech utterances aligned to French text) and contains linguists' transcriptions in Mboshi in the form of a non-standard graphemic 3 The dataset is documented in [13] and available at https://github.com/besacier/ mboshi-french-parallel-corpus 4 http://lig-aikuma.imag.fr…”

“…However, for many languages of the world, text transcripts are limited or nonexistent; therefore, recent efforts have been devoted to Zero Resource Settings [1,2,3] where the aim is to build speech systems without textual or linguistic resources for e.g. : (1) unwritten languages [4,5]; (2) models that mimic child language development [6]; (3) documentation of endangered languages by analyzing speech recordings using automatically discovered linguistic units (phones, morphs, words, etc) [7]. This paper focuses on unsupervised word segmentation from speech: the system must output time-stamps delimiting stretches of speech, associated with class labels, corresponding to real words in the language.…”

Attention-Passing Models for Robust and Data-Efficient End-to-End Speech Translation

“…Prior work noted that in severe low-resource situations it may actually be easier to collect speech paired with translations than transcriptions(Duong et al, 2016). However, we focus on well-resourced languages for which ASR and MT corpora exist and for which it is more realistic to obtain good speech translation accuracy.…”