Tensor Decomposition and Non-linear Manifold Modeling for 3D Head Pose Estimation

Abstract

Head pose estimation is a challenging computer vision problem with important applications in different scenarios such as human-computer interaction or face recognition. In this paper, we present a 3D head pose estimation algorithm based on non-linear manifold learning. A key feature of the proposed approach is that it allows modeling the underlying 3D manifold that results from the combination of rotation angles. To do so, we use tensor decomposition to generate separate subspaces for each variation factor and show that each of them has a clear structure that can be modeled with cosine functions from a unique shared parameter per angle. Such representation provides a deep understanding of data behavior. We show that the proposed framework can be applied to a wide variety of input features and can be used for different purposes. Firstly, we test our system on a publicly available database, which consists of 2D images and we show that the cosine functions can be used to synthesize rotated versions from an object from which we see only a 2D image at a specific angle. Further, we perform 3D head pose estimation experiments using other two types of features: automatic landmarks and histogram-based 3D descriptors. We evaluate our approach on two publicly available databases, and demonstrate that angle estimations can be performed by optimizing the combination of these cosine functions to achieve state-of-the-art performance.

Tensor Methods in Computer Vision and Deep Learning

“…Such factor matrices are employed next for extraction of invariant features. Similar analysis has also been done in analysis of motion signatures [186] and gait sequences [187], disentangling pose and body shape [188], [189], 3D head pose estimation [190], face transfer [191], appearance-based tracking [192], [193], structure from motion, and optical flow [194], [195]. A compositional variant of TensorFaces has also been proposed [196].…”

Tensor Methods in Computer Vision and Deep Learning

“…Such factor matrices are employed next for extraction of invariant features. Similar analysis has also been done in analysis of motion signatures [186] and gait sequences [187], disentangling pose and body shape [188], [189], 3D head pose estimation [190], face transfer [191], appearance-based tracking [192], [193], structure from motion, and optical flow [194], [195]. A compositional variant of TensorFaces has also been proposed [196].…”

Research on Student Classroom Attention Detection Based on Multi-Feature Fusion

“…When the EAR value is lower than the detection threshold, which is denoted as etv (eyes threshold value), the eye is considered to be closed. For a given frame in a video stream, the eye closure detection condition can be determined as shown in formula (2).…”

“…Fully mining and analyzing students' learning status data and conducting personalized teaching can stimulate students' enthusiasm for learning and thinking and improve learning outcomes.For the determination of learning focus, Gupta et al [1] analyze students' learning investment through four different emotions displayed by students in the classroom: high positive, low positive, high negative, and low negative. Derkach et al [2] identify student learning states by recognizing head postures. Psaltis et al [3] determine students' learning focus in teaching activities based on eye movements.…”