Occlusion Robust Face Recognition Based on Mask Learning With Pairwise Differential Siamese Network

Abstract

Deep Convolutional Neural Networks (CNNs) have been pushing the frontier of the face recognition research in the past years. However, existing general CNN face models generalize poorly to the scenario of occlusions on variable facial areas. Inspired by the fact that a human visual system explicitly ignores occlusions and only focuses on nonoccluded facial areas, we propose a mask learning strategy to find and discard the corrupted feature elements for face recognition. A mask dictionary is firstly established by exploiting the differences between the top convoluted features of occluded and occlusion-free face pairs using an innovatively designed Pairwise Differential Siamese Network (PDSN). Each item of this dictionary captures the correspondence between occluded facial areas and corrupted feature elements, which is named Feature Discarding Mask (FDM). When dealing with a face image with random partial occlusions, we generate its FDM by combining relevant dictionary items and then multiply it with the original features to eliminate those corrupted feature elements. Comprehensive experiments on both synthesized and realistic occluded face datasets show that the proposed approach significantly outperforms the state-of-the-arts.

Efficient Masked Face Recognition Method during the COVID-19 Pandemic

“…We nd deep learning based method based on the fact that human visual system automatically ignores the occluded regions and only focuses on the non-occluded ones. For example, Song et al [16] proposed a mask learning technique in order to discard the feature elements of the masked region for the recognition process.…”

Masked Face Recognition with Latent Part Detection

“…These methods are designed based on the hand-craft low-level features and the discriminative ability is limited. Based on the deep CNN, a multi-stage mask learning strategy is proposed in [35] to find and discard corrupted feature elements from recognition, which Figure 3: The procedure of mask generation. Firstly, we detect landmarks of the face image and locate the bounding box of the mask region.…”

Classification of Occluded Images for Large-Scale Datasets With Numerous Occlusion Patterns

“…In the deep feature space, the correspondence between the occluded image regions and the corrupted elements of the deep feature map, training with data augmentation, and recovering uncorrupted DFVs were investigated to improve the robustness of deep features to the occlusion. In [23], a pairwise differential Siamese network (PDSN) was proposed to learn the correspondence between the occluded facial regions and the corrupted activations of the top convolution layer. The occlusion-associated feature elements, which will be discarded in classification, are indicated by a discarding mask that is generated according to the learned correspondence.…”

“…Under the deep learning framework, occlusion can be tackled in a low-level representation, e.g. the image itself [16]- [22] or low-level deep feature map [23], or a high-level representation, e.g. the deep feature vector (DFV) [24], of the image.…”