Fully End-to-End Composite Recurrent Convolution Network for Deformable Facial Tracking In The Wild

Abstract

Human facial tracking is an important task in computer vision, which has recently lost pace compared to other facial analysis tasks. The majority of current available tracker possess two major limitations: their little use of temporal information and the widespread use of handcrafted features, without taking full advantage of the large annotated datasets that have recently become available. In this paper we present a fully end-to-end facial tracking model based on current state of the art deep model architectures that can be effectively trained from the available annotated facial landmark datasets. We build our model from the recently introduced general object tracker Re 3 , which allows modeling the short and long temporal dependency between frames by means of its internal Long Short Term Memory (LSTM) layers. Facial tracking experiments on the challenging 300-VW dataset show that our model can produce state of the art accuracy and far lower failure rates than competing approaches. We specifically compare the performance of our approach modified to work in tracking-by-detection mode and showed that, as such, it can produce results that are comparable to state of the art trackers. However, upon activation of our tracking mechanism, the results improve significantly, confirming the advantage of taking into account temporal dependencies.

End-to-end Facial and Physiological Model for Affective Computing and Applications

“…Since they may contain other redundant parts of the scene which consequently slow down the training process. To do this, we used facial tracking model of [37] and cropped facial area given detected facial landmark.…”

Latent-Based Adversarial Neural Networks for Facial Affect Estimations

“…On the Sentiment Analysis in the Wild dataset (SEWA) [23], we followed original person-independence protocols, and apply the feature extraction techniques described on previous sections. Moreover, we also use the external tracker of [2] to refine the given bounding box. We also includes the experiments on Aff-Wild2 dataset as part of the Affective Behavior Analysis in-the wild (ABAW) 2020 Competition to provide more actual analysis of our models performance.…”

Heatmap-Guided Balanced Deep Convolution Networks for Family Classification in the Wild

“…2) Facial Heatmap Estimator: F HE augments the input features by introducing an additional facial heatmap layer centered around the landmark estimates of l t [6]. To do this, we first obtain the facial landmark points l t using the F LL part of Recurrent Tracker [4], which consist of Inception-Resnet (res) [27] and a regression layer of weight matrix W F LL parameterized by Φ 3 :…”