Fine-Grained Head Pose Estimation Without Keypoints

Abstract

Estimating the head pose of a person is a crucial problem that has a large amount of applications such as aiding in gaze estimation, modeling attention, fitting 3D models to video and performing face alignment. Traditionally head pose is computed by estimating some keypoints from the target face and solving the 2D to 3D correspondence problem with a mean human head model. We argue that this is a fragile method because it relies entirely on landmark detection performance, the extraneous head model and an ad-hoc fitting step. We present an elegant and robust way to determine pose by training a multi-loss convolutional neural network on 300W-LP, a large synthetically expanded dataset, to predict intrinsic Euler angles (yaw, pitch and roll) directly from image intensities through joint binned pose classification and regression. We present empirical tests on common in-the-wild pose benchmark datasets which show state-of-the-art results. Additionally we test our method on a dataset usually used for pose estimation using depth and start to close the gap with state-of-the-art depth pose methods. We open-source our training and testing code as well as release our pre-trained models 1 .

Towards Real-Time Head Pose Estimation: Exploring Parameter-Reduced Residual Networks on In-the-wild Datasets

“…Other approaches evaluated on the AFLW and AFW datasets are summarized in Table 3. The number of parameters of [27] is based on their provided open source implementation, which is executable on a GPU based system. 4 In order to compare the frame rate, we reimplemented the LeNet-5 variant of [25].…”

“…4 In order to compare the frame rate, we reimplemented the LeNet-5 variant of [25]. In comparison, our ResNet18-64 has the lowest number of parameters while predicting more accurately than the LeNet-5 variant [25] and nearly as accurate as the ResNet50 [27]. Patacchiola and Cangelosi [25] also use low-resolution images with 64 x 64 pixels, while Ruiz et al [27] take larger images with 224 x 224 pixels.…”

“…There are several recent approaches that employ the in-the-wild datasets AFLW and AFW for training and use the AFW dataset as a testing benchmark: Patacchiola and Cangelosi [25] compare various LeNet-5 [19] variants trained with different gradient and adaptive gradient methods. Ruiz et al [27] train a 50-layer ResNet with a combined loss function of mean squared loss and cross entropy loss for all three angles. They achieve good results and outperform Patacchiola and Cangelosi.…”

Gaze360: Physically Unconstrained Gaze Estimation in the Wild

“…In this section, we compare several approaches using the Gaze360 dataset. We compared the following methods: Mean -uses the mean gaze of the training set for all predictions; Deep Head Pose -a deep network based head pose estimator by Ruiz et al [19]; Static -the backbone model, ResNet-18, and two final layers to compute the prediction; TRN -a version of Temporal Relation Network [33] where the features of frames at fixed windows around time t are concatenated before averaging the predictions of the temporal windows; LSTM -refers to the Gaze360 architecture.…”

Tree-Gated Deep Mixture-of-Experts for Pose-Robust Face Alignment

“…Following [16] we use a truncated pre-trained ResNet-50 network to extract head pose from the raw face image I. We note φ the embeddings (2048 units) of the last fullyconnected layer.…”