Tongue Contour Tracking and Segmentation in Lingual Ultrasound for Speech Recognition: A Review

Abstract

Lingual ultrasound imaging is essential in linguistic research and speech recognition. It has been used widely in different applications as visual feedback to enhance language learning for non-native speakers, study speech-related disorders and remediation, articulation research and analysis, swallowing study, tongue 3D modelling, and silent speech interface. This article provides a comparative analysis and review based on quantitative and qualitative criteria of the two main streams of tongue contour segmentation from ultrasound images. The first stream utilizes traditional computer vision and image processing algorithms for tongue segmentation. The second stream uses machine and deep learning algorithms for tongue segmentation. The results show that tongue tracking using machine learning-based techniques is superior to traditional techniques, considering the performance and algorithm generalization ability. Meanwhile, traditional techniques are helpful for implementing interactive image segmentation to extract valuable features during training and postprocessing. We recommend using a hybrid approach to combine machine learning and traditional techniques to implement a real-time tongue segmentation tool.

TongueTransUNet: Toward Effective Tongue Contour Segmentation Using Small Dataset

“…Tongue contour tracking has been explored using different techniques that are based on traditional computer vision techniques or deep learning-based algorithms [4]. Traditional techniques are based on two base models: The Snake algorithm, the active contour model, and the shape consistency model.…”

“…The head and oral cavity picture was modified from the original picture for the case, courtesy of Associate Professor Frank Gaillard, Radiopaedia.org, rID: 35836, [25]. Right: Output of recorded ultrasound image, ultrasound image source [18].…”

“…In the cited work [18], the researchers proposed a multi-stage algorithm that incorporates preprocessing to enhance noisy ultrasound images. The algorithm also extracts features using a combined shock filter and curvelet transform.…”

“…The methods that are based on Convolutional neural networks (CNNs) were the most efficient compared to other techniques. In the era of deep learning, UNet architecture became the stateof-the-art architecture to segment medical images [2][3][4]. UNet architecture is structured based on a stack of convolutional layers to form encoders and decoders arrangement [5].…”

Super-Resolved Dynamic 3D Reconstruction of the Vocal Tract during Natural Speech

“…MRI has become the gold standard modality for speech imaging [1]. It demonstrates numerous advantages over other imaging techniques such as electromagnetic articulography [2], X-rays [3], and ultrasound imaging [4,5]. However, due to physical constraints, it remains relatively slow, which complicates imaging of fast movements and leads to the presence of obstructing motion artifacts.…”