A Deep Learning Approach for Data Driven Vocal Tract Area Function Estimation

Abstract

In this paper we present a data driven vocal tract area function (VTAF) estimation using Deep Neural Networks (DNN). We approach the VTAF estimation problem based on sequence to sequence learning neural networks, where regression over a sliding window is used to learn arbitrary non-linear one-tomany mapping from the input feature sequence to the target articulatory sequence. We propose two schemes for efficient estimation of the VTAF; (1) a direct estimation of the area function values and (2) an indirect estimation via predicting the vocal tract boundaries. We consider acoustic speech and phone sequence as two possible input modalities for the DNN estimators. Experimental evaluations are performed over a large data comprising acoustic and phonetic features with parallel articulatory information from the USC-TIMIT database. Our results show that the proposed direct and indirect schemes perform the VTAF estimation with mean absolute error (MAE) rates lower than 1.65 mm, where the direct estimation scheme is observed to perform better than the indirect scheme.

Vocal Tract Contour Tracking in rtMRI Using Deep Temporal Regression Network

“…While these snapshots indicate that the proposed DTRN+AMCR model well captures the tract boundaries in all regions and under different articulatory configurations, a possible AoI classification error is also visible in the first snapshot due to a challenging calculation for the closest point of the contour to the landmark coordinate. Video samples of the tracked MRI contours are also available for online demonstration 3 .…”

“…In this context, development of automatic algorithms to detect the landmarks defining the contours of the vocal tract is necessary. To name a few applications, VT contour estimation is used as a preprocessing step to obtain and study the time evolution of the vocal tract cross sectional area function in [3]. In another work, Toutios et al [4] propose a text-to-speech synthesis system using the estimated VT contours in rtMRI.…”