DNN-based Acoustic-to-Articulatory Inversion using Ultrasound Tongue Imaging

Porras, Dagoberto; Sepúlveda-Sepúlveda, Alexander; Csapó, Tamás Gábor

doi:10.48550/arxiv.1904.06083

Cited by 1 publication

(2 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…MOCHA-TIMIT 3 is a database that contains EMA and acoustic data for 460 utterances (20 min) read by two English speakers [18]; USC-TIMIT 4 [19] provides EMA data for four speakers on the MOCHA-TIMIT sentences (15 min); and EMA-IEEE 5 [20] contains eight speakers reading 720 sentences, once each at a normal rate, and once at a fast speech rate (47 min per speaker). 6 The combined duration is 461 min.…”

Section: Descriptionmentioning

confidence: 99%

“…Acoustic-to-articulatory inversion is the problem of finding a mapping from acoustic features to a set of articulatory measures (see [1,2,3,4,5,6] for recent models: see [7] for a review). Reconstructed articulatory trajectories have been shown to improve text-to-speech synthesis [8,7], speech accent conversion [9], and automatic speech recognition [10,4], in particular for dysarthric speech [11]; they can also be used in the automatic detection of clinical conditions which have an impact on speech production, such as Parkinson's [12].…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Independent and automatic evaluation of acoustic-to-articulatory inversion models

Parrot,

Millet,

Dunbar

2019

Preprint

View full text Add to dashboard Cite

Reconstruction of articulatory trajectories from the acoustic speech signal has been proposed for improving speech recognition and text-to-speech synthesis. However, to be useful in these settings, articulatory reconstruction must be speaker independent. Furthermore, as most research focuses on single, small datasets with few speakers, robust articulatory reconstrucion could profit from combining datasets. Standard evaluation measures such as root mean square error and Pearson correlation are inappropriate for evaluating the speaker-independence of models or the usefulness of combining datasets. We present a new evaluation for articulatory reconstruction which is independent of the articulatory data set used for training: the phone discrimination ABX task. We use the ABX measure to evaluate a Bi-LSTM based model trained on 3 datasets (14 speakers), and show that it gives information complementary to the standard measures, and enables us to evaluate the effects of dataset merging, as well as the speaker independence of the model.

show abstract

Section: Descriptionmentioning

confidence: 99%