Soft biometric trait classification from real-world face videos conditioned on head pose estimation

Abstract

Recently, soft biometric trait classification has been receiving more attention in the computer vision community due to its wide range of possible application areas. Most approaches in the literature have focused on trait classification in controlled environments, due to the challenges presented by real-world environments, i.e. arbitrary facial expressions, arbitrary partial occlusions, arbitrary and nonuniform illumination conditions and arbitrary background clutter. In recent years, trait classification has started to be applied to real-world environments, with some success. However, the focus has been on estimation from single images or video frames, without leveraging the temporal information available in the entire video sequence. In addition, a fixed set of features are usually used for trait classification without any consideration of possible changes in the facial features due to head pose changes. In this paper, we propose a temporal, probabilistic framework first to robustly estimate continuous head pose angles from realworld videos, and then use this pose estimate to decide on the appropriate set of frames and features to use in a temporal fusion scheme for soft biometric trait classification. Experiments performed on large, real-world video sequences show that our head pose estimator outperforms the current state-of-the-art head pose approaches (by up to 51%), whereas our head pose conditioned biometric trait classifier (for the case of gender classification) outperforms the current state-of-the-art approaches (by up to 31%).

Speaker trait characterization in web videos: Uniting speech, language, and facial features

“…Linguistic features show significant correlation with the speaker attributes, yet their usefulness in practice might be limited by erroneous ASR. Naturally, also in the extraction of acoustic and facial features there is large room for improvement regarding robustness (for example, by compensation of channel effects, head pose estimation [11,18] or advanced face tracking, cf. [24]).…”

“…Many uni-modal approaches exist for classification of one or more of the person traits addressed in this paper: For instance, [9][10][11][12] use video, [13][14][15] use text, and [1,3,6] use audio only. Audio-visual approaches are presented in [16,17], but only for the gender task.…”

“…Especially gender recognition from static images has been thoroughly investigated (cf. [8] for an overview), and recently attention has shifted to videos [9][10][11]. Age and ethnicity / biological race have received less attention, but first studies report resonable performance in lab conditions [12] and real-life surveillance videos [10].…”

Age and Gender Estimation of Unfiltered Faces

“…Gender classification has received considerable attention over the years, both for its potential contribution to face recognition [3], as well as its applications in human computer interaction, soft biometrics [36], [37] and more. For a rigorous survey of the methods developed for this problem over the years, we refer to [38] or, the more recent [39].…”

From Clothing to Identity: Manual and Automatic Soft Biometrics

“…Nevertheless, soft comparative labels have been demonstrated to be more successful in representing the slight differences between people in bodily descriptions [23]. Facial marks, for instance, can be automatically detected and described to be used as micro-soft traits to supplement primary facial features for improved face recognition and fast retrieval; besides, they may enable matching with low resolution or partial images [29]. Measured facial information might be useful for gender prediction [30] and many system issues and challenges could arise when soft facial traits are used at a distance [20].…”