Head Pose Estimation in Complex Environment Based on Four-Branch Feature Selective Extraction and Regional Information Exchange Fusion Network

Abstract

Under the severe situation of the COVID-19 pandemic, masks cover most of the effective facial features of users, and their head pose changes significantly in a complex environment, which makes the accuracy of head pose estimation in some systems such as safe driving systems and attention detection systems impossible to guarantee. To this end, we propose a powerful four-branch feature selective extraction network (FSEN) structure, in which three branches are used to extract three independent discriminative features of pose angles, and one branch is used to extract composite features corresponding to multiple pose angles. By reducing the dimension of high-dimensional features, our method significantly reduces the amount of computation while improving the estimation accuracy. Our convolution method is an improved spatial channel dynamic convolution (SCDC) that initially enhances the extracted features. Additionally, we embed a regional information exchange network (RIEN) after each convolutional layer in each branch to fully mine the potential semantic correlation between regions from multiple perspectives and learn and fuse this correlation to further enhance feature expression. Finally, we fuse the independent discriminative features of each pose angle and composite features from the three directions of channel, space, and pixel to obtain perfect feature expression for each pose angle, and then obtain the head pose angle. We conducted extensive experiments on the controlled environment datasets and a self-built real complex environment dataset (RCE) and the results showed that our method outperforms state-of-the-art single-modality methods and performs on par with multimodality-based methods. This shows that our network meets the requirements of accurate head-pose estimation in real complex environments such as complex illumination and partial occlusion.INDEX TERMS Head pose estimation, four-branch feature selective extraction, regional information exchange network, spatial channel dynamic convolution, multiple feature fusion, complex environment.

Deep Learning for Head Pose Estimation: A Survey

“…Not only bounding box affect the final result but also illumination and occlusion, for this reason Wang et al in their FSEN [154] included low light enhancement, strong light suppression and face occlusion detection modules. This united with a four-branch CNN, in which three branches are used to extract three independent discriminative features of pose angles, and one branch is used to extract composite features corresponding to multiple pose angles, improved the results on benchmark datasets.…”