A multimodal face antispoofing method based on multifeature vision transformer and multirank fusion

Abstract

SummaryFace antispoofing (FAS) is attracting increasing attention from researchers because of its important role in preventing facial recognition systems from face spoofing attacks. With the advancement of various new convolutional neural network structures and the construction of various face antispoofing databases, the deep learning‐based face antispoofing algorithm has become the main method in the FAS field. However, the generalization performance of the current multimodal face antispoofing algorithm is poor, and the recognition performance of the model on different datasets is quite different. Therefore, we design a multimodal face antispoofing framework based on a multifeature transformer (MFViT) and multirank fusion (MRF). First, we use a vision transformer structure, MFViT, for multimodal face antispoofing and a combination of modalities to capture the distinguishing characteristics in each modality. Second, we design a multidimensional multimodal fusion module, MRF, according to the various modal fusion characteristics obtained by the MFViT to fuse modal information in different dimensions more effectively. Evaluation results indicate that framework we designed achieves an average classification error rate (ACER) of 1.61% on the CASIA‐SURF dataset and an ACER of 6.5% on the CASIA‐SURF CeFA dataset.