PipeNet: Selective Modal Pipeline of Fusion Network for Multi-Modal Face Anti-Spoofing

Yang, Qing; Zhu, Xiaoyan; Fwu, Jong-Kae; Ye, Yun; You, Ganmei; Zhu, Yuan

doi:10.1109/cvprw50498.2020.00330

Cited by 32 publications

(12 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…proposed in [141] the combination of spatial and spectral features. Another two interesting fusion approaches have been recently presented in [192], [193], combining RGB, Depth, and InfraRed information to detect physical face attacks. Also, face weighting approaches have been proposed in order to detect fake videos using multiple frames [194].…”

Section: Discussionmentioning

confidence: 99%

Deepfakes and beyond: A Survey of face manipulation and fake detection

et al. 2020

View full text Add to dashboard Cite

Section: Discussionmentioning

confidence: 99%

Deepfakes and beyond: A Survey of face manipulation and fake detection

et al. 2020

View full text Add to dashboard Cite

“…They also proposed a novel static-dynamic fusion mechanism as a strong baseline to learn complementary information from multiple modalities. To further differentiate feature extraction among different modality-data, Yang et al [30] designed a selective modal pipeline of fusion network for multi-modal face anti-spoofing. However, aforementioned works are evaluated on a small face anti-spoofing dataset with limited subjects and samples, which easily leads to the problem of over-fitting.…”

Section: A Face Anti-spoofingmentioning

confidence: 99%

Data-Fusion-Based Two-Stage Cascade Framework for Multimodality Face Anti-Spoofing

Liu

Wei²,

Lei

et al. 2022

IEEE Trans. Cogn. Dev. Syst.

View full text Add to dashboard Cite

Existing face anti-spoofing models using deep learning for multi-modality data suffer from low generalization in the case of using variety of presentation attacks such as 2D printing and high-precision 3D face masks. One of the main reasons is that the non-linearity of multi-spectral information used to preserve the intrinsic attributes between a real and a fake face are not well extracted. To address this issue, we propose a multi-modility data based two-stage cascade framework for face anti-spoofing. The proposed framework has two advantages. Firstly, we design a two-stage cascade architecture that can selectively fuse lowlevel and high-level features from different modalities to improve feature representation. Secondly, we use multi-modality data to construct a distance-free spectral on RGB and infrared (IR) to augment the non-linearity of data. The presented data fusion strategy is different from popular fusion approaches, since it can strengthen discrimination ability of network models on physical attribute features than identity structure features under certain constraints. In addition, a multi-scale patch based weighted finetuning strategy is designed to learn each specific local face region. Experimental results show that the proposed framework achieves better performance than other state-of-the-art methods on both benchmark datasets and self-established datasets, especially on multi-material masks spoofing.

show abstract

“…Under a simple live/spoof binary classification settings, resulting two representations separated by a classification boundary can have mixed information, such as subject ID. Several neural networks are proposed to disentangle live/spoof class signature from the unwanted information [48], [49], [50] Because most of the presentation attacks are conducted under visible-light domain, using multimodal input or non-visible modality (e.g., depth, rPPG, SWIR) are useful for increasing PAD performance [6], [51], [52], [53], [54], [55], [56], [57], [58], [59], [60], [61], [62].…”

Section: Dnnsmentioning

confidence: 99%