Lip segmentation using automatic selected initial contours based on localized active contour model

Abstract

With the rapid development of artificial intelligence and the increasing popularity of smart devices, human-computer interaction technology has become a multimedia and multimode technology from being computer-focused to people-centered. Among all ways of human-computer interactions, using language to interact with machines is the most convenient and efficient one. However, the performance of audio speech recognition systems is not satisfied in a noisy environment. Thus, more and more researchers focus their works on visual lip reading technology. By extracting lip movement features of speakers rather than audio features, visual lip reading systems can get superior results when noises and interferences exist. Lip segmentation plays an important role in a visual lip reading system, since the segmentation result is crucial to the final recognition accuracy. In this paper, we propose a localized active contour model-based method using two initial contours in a combined color space. We apply illumination equalization to original RGB images to decrease the interference of uneven illumination. A combined color space consists of the U component in CIE-LUV color space and the sum of C2 and C3 components of the image after discrete Hartley transform. We select a rhombus as the initial contour of a closed mouth, because it has a similar shape to a closed lip. For an open mouth, we utilize a combined semi-ellipse as the initial contours of both outer and inner lip boundaries. After attaining the results of each color component separately, we merge them together to obtain the final segmentation result. From the experiment, we can conclude that this method can get better segmentation results compared with the method using a circle as the initial contour to segment gray images and images in combined color space, especially for open mouth. An extremely obvious advantage of this method is the results of open mouth excluding internal information of mouth such as teeth, black holes, and tongue, because of the introduction of the inner initial contour.

Realistic Smile Expression Recognition Approach Using Ensemble Classifier with Enhanced Bagging

“…The proposed facial expression recognition system is tested with benchmark datasets that includes Japanese Female Facial Expressions (JAFFE), Extended Cohn-Kanade (CK+), CK+48 Dataset [46]. JAFFE is a Japanese database containing 7 facial expressions with a 256 × 256pixel resolution of 213 images.…”

“…( 2 The base classifiers execute classification operations, and the final prediction is rendered using plurality voting on the ensemble subset outputs. If classifiers dispute, the voting process will be used to exclude the various classifiers' incorrect mistakes [45][46][47][48]. Fig.…”

Visual Speech Recognition using VGG16 Convolutional Neural Network

“…The author proposed a limited active contour model-based technique used to segment the lip area. Lip separation is essential to graphic lip-reading systems, because the precision of segmentation result directly affects the recognition rate [22]. a model called Watch, Listen, Attend and spell commonly called as WLAS model was contributed.…”

Efficient Scale Invariant and Back Propagation Neural Network Method using LIP Region Segmentation

“…Lu Y and Liu, Q et al, 2018 [16] proposed research on local active contour method utilizing basic contour by linking color areas. A brilliance parallelization applied towards RGB pictures for reducing the distortion.…”