Computational efficient deep neural network with difference attention maps for facial action unit detection

Abstract

There are more and more applications of attention mechanism in the field of computer vision. As a task in the field of vision, facial action unit detection has gradually used attention mechanism. Attention mechanism can make the network pay more attention to specific areas to capture the features related to AU. Most of the existing attention-based methods use facial landmark detection to predefine the regions associated with AU. These methods depend on the accuracy of facial landmark detection. Region learning is not conducive to learning the correlation between AUs. The number of predefined regions of interest is generally fixed. Processing parts that have nothing to do with the target task also increases the computational burden and network complexity. In this paper, we propose a computational efficient end-to-end training deep neural network (CEDNN) model and spatial attention maps based on difference images. Firstly, the difference image is generated by image processing. Then five binary images of difference images are obtained by using different thresholds. These binary images are regarded as spatial attention maps. We use the group convolution to reduce model complexity. Skip connection and 1 × 1 convolution are used to ensure good performance in the case of low depth. As an input, spatial attention map can be selectively fed into the input of each block. In addition, we only need to adjust the parameters of the classifier to easily extend to other datasets without