Face and Facial Expressions Recognition System for Blind People Using ResNet50 Architecture and CNN

Abstract

Many blind individuals have difficulties in recognizing people’s facial expression which may impact their social interaction. With the recognition, the blind individuals can accurately interpret and respond to the emotions. There is a lack in the existing application with the combination of face and facial expressions recognition. The blind individuals have to rely on multiple applications to accomplish the same task, making it difficult and time-consuming for them to use. The paper aims to recognize faces and facial expressions for blind individuals and provides feedback in real-time. Three face detection algorithms of Haar Cascade Classifier, Dlib, and RetinaFace are compared. Dlib is chosen to process with Histogram of Oriented Gradients (HOG) and Support Vector Machine (SVM). It loads the pre-trained model, computes the HOG features, slide the window scanning at different scales, classify the windows using the SVM classifier, generate bounding boxes, and applying non-maximum suppression. ResNet50 architecture is employed to recognize face and Convolutional Neural Networks (CNN) is applied to recognize facial expression. The training accuracy is 70% and validation accuracy is 60%.

Deep convolutional neural network for weld defect classification in radiographic images

“…ResNet50 is a very effective neural network architecture for image classification and other computer vision problems [ 28 , [70] , [71] , [72] , [73] , [74] ]. Being pre-trained on large image datasets, ResNet50 can provide high performance without the need to train a neural network from scratch; it is further used to save training time and to aid the effective generalization of the models built by learning from visual patterns and features of the image set it was initially trained on.…”

Real-Time Object Detection and Face Recognition Application for the Visually Impaired

“…We also explored other methods such as Convolutional Neural Network (CNN) based face recognition [25], which demands extensive data and computational resources; LBPH method [26], which produces long histograms leading to slower recognition speeds; and the EigenFace [27] algorithm, which is sensitive to lighting variations. This study [28] uses the "dlib" library, employing Histogram Oriented Gradients (HOG) for feature extraction and Support Vector Machines (SVM) for differentiating face and non-face regions. Notably, the "dlib" method is solely used for identifying face bounding boxes, while a Deep Neural Network-based classification model categorizes the detected faces.…”

COVID-19 Social Distancing Tracking and Monitoring System (SDMOS-19)

“…A CNN is an artificial neural network explicitly designed to process pixels for image and video recognition [22] [23]. CNNs are powered by AI techniques implemented in image and video processing systems to operate conceptual and insightful duties [24].…”