A Comprehensive Experimental and Reproducible Study on Selfie Biometrics in Multistream and Heterogeneous Settings

Abstract

This contribution presents a new database to address current challenges in face recognition. It contains face video sequences of 75 individuals acquired either through a laptop webcam or when mimicking the front-facing camera of a smartphone. Sequences have been acquired with a device allowing to record visual, near-infrared and depth data at the same time. Recordings have been made across three sessions with different, challenging illumination conditions and variations in pose. Together with the database, several experimental protocols are provided and correspond to real world scenarios, when a mismatch in conditions between enrollment and probe images occurs. A comprehensive set of baseline experiments using publicly available baseline algorithms show that extreme illumination conditions and pose variations are remaining issues. However, the usage of different data domains-and their fusion-allows to mitigate such variation. Finally, experiments on heterogeneous face recognition are also presented using a state-of-the-art model based on deep neural networks, and showed better performance. When applied to other tasks, this model proved to surpass all existing baselines as well. The data, as well as the code to reproduce all experiments are made publicly available to help foster research in selfie biometrics using latest imaging devices.

Data-Fusion-Based Two-Stage Cascade Framework for Multimodality Face Anti-Spoofing

“…Compared to general RGB cameras that only provide three channels from visible wavelength bands, multispectral imagers provide more broadly wavelength bands, which can greatly resist on the interference of illumination and reflect more intrinsic object attributes [19][20][21]. Typically, reflectance images of real faces show low intensity and uniform distribution while reflectance images of 2D/3D spoofing faces show high intensity and non-uniform distribution [22,23]. In the past few years, few studies have been devoted to multi-spectral based face anti-spoofing due to the lack of portable devices.…”

Remote photoplethysmography (rPPG) in the wild: Remote heart rate imaging via online webcams.

“…We tested the rPPG algorithm on the CohFace video dataset (Heusch, Anjos et al 2017) because the video properties and light conditions most closely match those of the videos recorded in Study 2. The CohFace dataset includes 164 videos from 40 individuals.…”

“…Suboptimal conditions may severely impair the possibility of using rPPG techniques in out-of-lab environments, such as during online data collection, where researchers cannot control the setting beforehand, the illumination, or the hardware used by the participants. While several video databases for rPPG benchmarking are openly available (Soleymani, Lichtenauer et al 2012, Stricker et al, 2014, Heusch et al, 2017, Bobbia, Macwan et al 2019, finding a way to include rPPG in online experiments could provide researchers with a tool that can be deployed to collect physiological parameters remotely, fostering largescale psychophysiological data collections outside the lab.…”

“…In two studies, we will demonstrate rPPG extraction merged with an online experimental platform that allows the recording of participants' heart rate despite the limited control over the context and the setup. In study 1, we first compared the results of a modified open-source rPPG extraction algorithm (van der Kooij and Naber 2019) to a validated dataset (CohFace) used for rPPG testing, assessing the convergence of the rPPG algorithm results versus bloodvolume pulse (BVP) data collected directly with a sensor from Thought Technologies (for further details see Heusch et al (2017)). In the second study, we collected 231 online video recordings of 18 participants and we compared the results of the rPPG extraction against finger pulse oximeter data, collected with a validated mobile heart rate application (Losa-Iglesias, Becerro-de-Bengoa-Vallejo et al 2016).…”

Adapting remote photoplethysmography for Indonesian subjects: an examination of diverse rPPG techniques

“…Research in the field of remote photoplethysmography (rPPG) has thus far relied heavily on datasets predominantly composed of subjects of Western or Oriental descent, such as the VIPL-HR [20], MAHNOB HCI [21], COHFACE [22], UBFC-rPPG [23], PURE [24], and OBF [25] datasets.…”