SmileNet: Registration-Free Smiling Face Detection In The Wild

Abstract

We present a novel smiling face detection framework called SmileNet for detecting faces and recognising smiles in the wild. SmileNet uses a Fully Convolutional Neural Network (FCNN) to detect multiple smiling faces in a given image of varying resolution. Our contributions are threefold: 1) SmileNet is the first smiling face detection network that does not require pre-processing such as face detection and registration in advance to generate a normalised (cropped and aligned) input image; 2) the proposed SmileNet is a simple and single FCNN architecture simultaneously performing face detection and smile recognition, which are conventionally treated as separate consecutive pipelines; and 3) SmileNet ensures real-time processing speed (21.15 FPS) even when detecting multiple smiling faces in a given image (300 × 300). Experimental results show that SmileNet can deliver state-of-the-art performance (95.76%), even under occlusions, and variances of pose, scale, and illumination.

Embedded Implementation of a Deep Learning Smile Detector

“…In full smile detection pipelines, such as the SmileNet [27], also a face localization step is included. Thus, in order to evaluate the networks in the context of an entire system, we attach the smile detection network together with the Viola-Jones face detector running on the CPU.…”

“…Several studies on detection of apparent age, gender and smile using deep learning methods have been published, e.g. [25], smile detection for image data of young children [26], and SmileNet [27] a modern network architecture for smile detection. A deep multi-task learning framework HyperFace can provide simultaneous face detection, landmarks localization, pose estimation and gender recognition [28].…”

SchiNet: Automatic Estimation of Symptoms of Schizophrenia from Facial Behaviour Analysis

“…SchiNet performs the analysis in 4 stages; preprocessing, lowlevel feature extraction at frame level, high-level feature extraction at video level and symptoms regression. At the first stage, we detect the patients' faces in the video frames using a body detector [27] and a robust face detector [28]. At the second stage, the face regions are cropped and passed to a bank of Deep Neural Networks (DNNs), each of which detects a certain facial expression or the activation of a certain facial Action Unit.…”

“…First, we detect the patient's body at each frame using the Single Shot Detector (SSD) proposed in [27]. We then extend the detected body-bounding box by a factor of 1.2 to ensure that the whole head is included, and then, within the resulting region, we apply SmileNet [28] to detecting the bounding box of the face and whether it is smiling or not. Finally, we crop and scale the detected face to a fixed resolution of 100×100 for further analysis.…”

“…In total, 11 facial expressions are analysed, ten of which are facial AUs detected using the AFEA method described above, and one is smile recognized using the SmileNet proposed in [28]. This results in an 11-dimensional feature vector for each frame where each dimension represents the probability of one of the detected expressions.…”