No abstract

Abstract-A system that could automatically analyze the facial actions in real time has applications in a wide range of different fields. However, developing such a system is always challenging due to the richness, ambiguity, and dynamic nature of facial actions. Although a number of research groups attempt to recognize facial action units (AUs) by improving either the facial feature extraction techniques or the AU classification techniques, these methods often recognize AUs or certain AU combinations individually and statically, ignoring the semantic relationships among AUs and the dynamics of AUs. Hence, these approaches cannot always recognize AUs reliably, robustly, and consistently. In this paper, we propose a novel approach that systematically accounts for the relationships among AUs and their temporal evolutions for AU recognition. Specifically, we use a dynamic Bayesian network (DBN) to model the relationships among different AUs. The DBN provides a coherent and unified hierarchical probabilistic framework to represent probabilistic relationships among various AUs and to account for the temporal changes in facial action development. Within our system, robust computer vision techniques are used to obtain AU measurements. Such AU measurements are then applied as evidence to the DBN for inferring various AUs. The experiments show that the integration of AU relationships and AU dynamics with AU measurements yields significant improvement of AU recognition, especially for spontaneous facial expressions and under more realistic environment including illumination variation, face pose variation, and occlusion.

No abstract

Over the past few years, Convolutional Neural Networks (CNNs) have shown promise on facial expression recognition. However, the performance degrades dramatically under real-world settings due to variations introduced by subtle facial appearance changes, head pose variations, illumination changes, and occlusions.In this paper, a novel island loss is proposed to enhance the discriminative power of the deeply learned features. Specifically, the IL is designed to reduce the intra-class variations while enlarging the inter-class differences simultaneously. Experimental results on four benchmark expression databases have demonstrated that the CNN with the proposed island loss (IL-CNN) outperforms the baseline CNN models with either traditional softmax loss or the center loss and achieves comparable or better performance compared with the state-of-the-art methods for facial expression recognition.

PurposeTo develop a new intelligent system based on deep learning for automatically optical coherence tomography (OCT) images categorization.MethodsA total of 60,407 OCT images were labeled by 17 licensed retinal experts and 25,134 images were included. One hundred one-layer convolutional neural networks (ResNet) were trained for the categorization. We applied 10-fold cross-validation method to train and optimize our algorithms. The area under the receiver operating characteristic curve (AUC), accuracy and kappa value were calculated to evaluate the performance of the intelligent system in categorizing OCT images. We also compared the performance of the system with results obtained by two experts.ResultsThe intelligent system achieved an AUC of 0.984 with an accuracy of 0.959 in detecting macular hole, cystoid macular edema, epiretinal membrane, and serous macular detachment. Specifically, the accuracies in discriminating normal images, cystoid macular edema, serous macular detachment, epiretinal membrane, and macular hole were 0.973, 0.848, 0.947, 0.957, and 0.978, respectively. The system had a kappa value of 0.929, while the two physicians' kappa values were 0.882 and 0.889 independently.ConclusionsThis deep learning-based system is able to automatically detect and differentiate various OCT images with excellent accuracy. Moreover, the performance of the system is at a level comparable to or better than that of human experts. This study is a promising step in revolutionizing current disease diagnostic pattern and has the potential to generate a significant clinical impact.Translational RelevanceThis intelligent system has great value in increasing retinal diseases' diagnostic efficiency in clinical circumstances.