Vocal Tract Airway Tissue Boundary Tracking for rtMRI Using Shape and Appearance Priors

Asadiabadi, Sasan; Erzin, Engin

doi:10.21437/interspeech.2017-1016

Cited by 11 publications

(5 citation statements)

References 6 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In another study, Asadiabadi and Erzin [20] propose a combination of ACM and ASM to overcome the shortcomings of the SNAKE algorithm in [11], when the articulators are in contact. In this study, an appearance model is built for the VT and is utilized as a contributing term to the energy minimization in the curve evolution process.…”

Section: Introductionmentioning

confidence: 99%

Vocal Tract Contour Tracking in rtMRI Using Deep Temporal Regression Network

Asadiabadi

Erzin

2020

IEEE/ACM Trans. Audio Speech Lang. Process.

Self Cite

View full text Add to dashboard Cite

Recent advances in real-time Magnetic Resonance Imaging (rtMRI) provide an invaluable tool to study speech articulation. In this paper, we present an effective deep learning approach for supervised detection and tracking of vocal tract contours in a sequence of rtMRI frames. We train a single input multiple output deep temporal regression network (DTRN) to detect the vocal tract (VT) contour and the separation boundary between different articulators. The DTRN learns the non-linear mapping from an overlapping fixed-length sequence of rtMRI frames to the corresponding articulatory movements, where a blend of the overlapping contour estimates defines the detected VT contour. The detected contour is refined at a post-processing stage using an appearance model to further improve the accuracy of VT contour detection. The proposed VT contour tracking model is trained and evaluated over the USC-TIMIT dataset. Performance evaluation is carried out using three objective assessment metrics for the separating landmark detection, contour tracking and temporal stability of the contour landmarks in comparison with three baseline approaches from the recent literature. Results indicate significant improvements with the proposed method over the state-of-the-art baselines.

show abstract

Section: Introductionmentioning

confidence: 99%

Vocal Tract Contour Tracking in rtMRI Using Deep Temporal Regression Network

Asadiabadi

Erzin

2020

IEEE/ACM Trans. Audio Speech Lang. Process.

Self Cite

View full text Add to dashboard Cite

show abstract

“…Unfortunately, the air-tissue boundary of the vocal tract, which is necessary for detailed analysis, is not always clearly identifiable because of issues such as noise. To overcome this problem, many intensive studies have been conducted to automatically or semi-automatically segment these tissues from the air [5][6][7][8][9]. A maximum temporal resolution of 100 frames per second (fps) was reached in 2015 [10], and as a large number of frames are available for use, such segmentation methods may become increasingly important.…”

Section: Introductionmentioning

confidence: 99%

Speech Organ Contour Extraction Using Real-Time MRI and Machine Learning Method

Takemoto¹,

Goto²,

Hagihara³

et al. 2019

Interspeech 2019

View full text Add to dashboard Cite

Real-time MRI can be used to obtain videos that describe articulatory movements during running speech. For detailed analysis based on a large number of video frames, it is necessary to extract the contours of speech organs, such as the tongue, semi-automatically. The present study attempted to extract the contours of speech organs from videos using a machine learning method. First, an expert operator manually extracted the contours from the frames of a video to build training data sets. The learning operators, or learners, then extracted the contours from each frame of the video. Finally, the errors representing the geometrical distance between the extracted contours and the ground truth, which were the contours excluded from the training data sets, were examined. The results showed that the contours extracted using machine learning were closer to the ground truth than the contours traced by other expert and non-expert operators. In addition, using the same learners, the contours were extracted from other naive videos obtained during different speech tasks of the same subject. As a result, the errors in those videos were similar to those in the video in which the learners were trained.

show abstract

“…For a successful image-based VTAF estimation, obtaining a full or semi-automatic vocal tract boundary segmentation is required. Intensive research has been conducted on the automatic vocal tract boundary estimation in the real-time MRI data [1,2,3,4]. Upon estimating the VT boundaries, the VTAF could be calculated in a number of ways.…”

Section: Introductionmentioning

confidence: 99%

A Deep Learning Approach for Data Driven Vocal Tract Area Function Estimation

Asadiabadi

Erzin

2018

2018 IEEE Spoken Language Technology Workshop (SLT)

Self Cite

View full text Add to dashboard Cite

In this paper we present a data driven vocal tract area function (VTAF) estimation using Deep Neural Networks (DNN). We approach the VTAF estimation problem based on sequence to sequence learning neural networks, where regression over a sliding window is used to learn arbitrary non-linear one-tomany mapping from the input feature sequence to the target articulatory sequence. We propose two schemes for efficient estimation of the VTAF; (1) a direct estimation of the area function values and (2) an indirect estimation via predicting the vocal tract boundaries. We consider acoustic speech and phone sequence as two possible input modalities for the DNN estimators. Experimental evaluations are performed over a large data comprising acoustic and phonetic features with parallel articulatory information from the USC-TIMIT database. Our results show that the proposed direct and indirect schemes perform the VTAF estimation with mean absolute error (MAE) rates lower than 1.65 mm, where the direct estimation scheme is observed to perform better than the indirect scheme.

show abstract

Vocal Tract Airway Tissue Boundary Tracking for rtMRI Using Shape and Appearance Priors

Cited by 11 publications

References 6 publications

Vocal Tract Contour Tracking in rtMRI Using Deep Temporal Regression Network

Vocal Tract Contour Tracking in rtMRI Using Deep Temporal Regression Network

Speech Organ Contour Extraction Using Real-Time MRI and Machine Learning Method

A Deep Learning Approach for Data Driven Vocal Tract Area Function Estimation

Contact Info

Product

Resources

About