Lip image segmentation based on a fuzzy convolutional neural network

Abstract

Research has shown that the human lip and its movements are a rich source of information related to speech content and speaker's identity. Lip image segmentation, as a fundamental step in many lipreading and visual speaker authentication systems, is of vital importance. Because of variations in lip color, lighting conditions and especially the complex appearance of an open mouth, accurate lip region segmentation is still a challenging task. To address this problem, this paper proposes a new fuzzy deep neural network having an architecture that integrates fuzzy units and traditional convolutional units. The convolutional units are used to extract discriminative features at different scales to provide comprehensive information for pixel-level lip segmentation. The fuzzy logic modules are employed to handle various kinds of uncertainties and to provide a more robust segmentation result. An end-to-end training scheme is then used to learn the optimal parameters for both the fuzzy and the convolutional units. A dataset containing more than 48,000 images of various speakers, under different lighting conditions, was used to evaluate lip segmentation performance. According to the experimental results, the proposed method achieves state-of-the-art performance when compared with other algorithms.

Deep Neuro-Fuzzy System application trends, challenges, and future perspectives: a systematic survey

“…In addition, the authors of (Deng et al 2017 ) presented an FDNN for classification tasks, such as natural scene image categorization and stock trend prediction. Similar studies for image classification can be found in (Guan et al 2020 ; Kunchala et al 2020 ; Liu et al 2020a , b ; Liu et al 2020a , b ; Manchanda et al 2020 ; Tianyu and Xu 2020 ; Yeganejou and Dick 2018 , 2019 ; Yeganejou et al 2020 ; Zhang et al 2020a , b , c ).…”

A Robust SVM Color-Based Food Segmentation Algorithm for the Production Process of a Traditional Carasau Bread

“…A CNN is a multilayer neural network made up of convolutional layers and pooling layers, whose neurons take small patches of the previous layer as input, and fully connected layers. CNNs are currently considered as one of the most popular machine intelligence model for big data analysis in various research areas; a number of CNN architectures with feedback mechanisms applied to image classification and image recognition have been proposed in literature [43], [44], [45], [46], [47].…”

Dynamic Video Image Segmentation Based on Dual Channel Convolutional Kernel and Multi-Frame Feature Fusion

“…In order to fill the image, the interior of the edge feature should be completely filled using the white color, and the two-dimensional binary image can be converted into a three-dimensional RGB image (Guan et al, 2019 ). To remain the color feature of the separated fire image as the same as the original image, the three-dimensional RGB image is merged with the original image, as shown in Figure 5F .…”