RECOApy: Data Recording, Pre-Processing and Phonetic Transcription for End-to-End Speech-Based Applications

Abstract

Deep learning enables the development of efficient end-to-end speech processing applications while bypassing the need for expert linguistic and signal processing features. Yet, recent studies show that good quality speech resources and phonetic transcription of the training data can enhance the results of these applications. In this paper, the RECOApy tool is introduced. RECOApy streamlines the steps of data recording and pre-processing required in end-to-end speech-based applications. The tool implements an easy-to-use interface for prompted speech recording, spectrogram and waveform analysis, utterance-level normalisation and silence trimming, as well grapheme-to-phoneme conversion of the prompts in eight languages: Czech, English, French, German, Italian, Polish, Romanian and Spanish.The grapheme-to-phoneme (G2P) converters are deep neural network (DNN) based architectures trained on lexicons extracted from the Wiktionary online collaborative resource. With the different degree of orthographic transparency, as well as the varying amount of phonetic entries across the languages, the DNN's hyperparameters are optimised with an evolution strategy. The phoneme and word error rates of the resulting G2P converters are presented and discussed. The tool, the processed phonetic lexicons and trained G2P models are made freely available.

RoLEX: The development of an extended Romanian lexical dataset and its evaluation at predicting concurrent lexical information

“…An encoder-decoder model with attention (Toshniwal and Livescu 2016) and a convolutional architecture combined with n-grams (Rao et al 2015) achieve similar results when applied to the same dataset. Transformer-based architectures are proposed in Sun et al (2019); Yolchuyeva, Németh, and Gyires-Tóth (2019b); Stan (2020) and slightly improve the error rates. Sun et al (2019) report a WER around 20% obtained with a model enriched through knowledge distillation using unlabelled source words.…”

“…The Transformer’s hyperparameter selection is based on the strategy introduced in (Stan 2020). The set of hyperparameters which were optimised are shown in Table 8.…”

“…For Romanian, the three lexical tasks were also subject to a series of studies covering Marcus Contextual Grammars (Dinu 2003), rule-based methods (Toma andMunteanu 2009), decision trees and linear models (Cucu et al 2014), cascaded sequential models Dinu et al 2014) or neural network-based approaches (Boroş, Dumitrescu, and Pais 2018;Stan and Giurgiu 2018;Stan 2019Stan , 2020. The reported WER for stress prediction is 2.36%, while the reported WERs for the phonemic transcription are between 1% and 3%.…”

An objective evaluation of the effects of recording conditions and speaker characteristics in multi-speaker deep neural speech synthesis

“…The average number of utterances pertaining to each speaker is 1747, and the total duration of the recordings for all speakers amounts to 59 hours and 39 minutes. Due to the current global pandemic situation, the recordings were performed in the speakers' home environments using the RECOApy tool [21], and were lightly checked for errors by the authors. Some of the issues notices in the recordings refer to the reverberation and background noise presence, as well as some utterances which are chopped either in the beginning or in the end, thus yielding incorrect text-to-audio alignments.…”