Impact of facial tattoos and paintings on face recognition systems

Abstract

In recent years, face recognition technologies have shown impressive recognition performance, mainly due to recent developments in deep convolutional neural networks. Notwithstanding those improvements, several challenges which affect the performance of face recognition systems remain. The authors investigate the impact that facial tattoos and paintings have on current face recognition systems. To this end, they first collected an appropriate database containing image-pairs of individuals with and without facial tattoos or paintings. The assembled database was used to evaluate how facial tattoos and paintings affect the detection, quality estimation, as well as the feature extraction and comparison modules of a face recognition system. The impact on these modules was evaluated using state-of-the-art open-source and commercial systems. The obtained results show that facial tattoos and paintings affect all the tested modules, especially for images where a large area of the face is covered with tattoos or paintings. The authors' work is an initial case study and indicates a need to design algorithms which are robust to the visual changes caused by facial tattoos and paintings.This is an open access article under the terms of the Creative Commons Attribution License, which permits use, distribution and reproduction in any medium, provided the original work is properly cited.

Face Beneath the Ink: Synthetic Data and Tattoo Removal with Application to Face Recognition

“…In coherence with the findings in [4], it is of interest to make facial analysis systems more robust towards facial tattoos. One way to do this is face completion, where missing or occluded parts of a face are reconstructed; such approaches have, for instance, shown to improve face recognition performance for some occlusions [5].…”

“…The database used by the authors includes different types of disguises including facial paintings. Coherent with these findings, Ibsen et al showed in [4] that facial tattoos and paintings can severely affect different modules of a face recognition system including face detection as well as feature extraction and comparison.…”

“…As part of the preprocessing, we propose to use the deep learning-based removal algorithms described in Sect. V. While facial tattoos can be seen as distinctive and helpful to identify individuals, tattoo removal is useful for face recognition in cases where only one of the face images in a comparison contains tattoos [4]. In our experiments, we trained the classifiers to remove facial tattoos from aligned images and as such will assume that our input images have already been aligned since our focus is on improving feature extraction and comparison.…”

“…In the following, we describe the database, the employed face recognition system, and metrics used to evaluate the biometric performance: Database for the evaluation we use the publicly available database HDA Facial Tattoo and Painting Database 3 , which consists of 250 image pairs of individuals with and without real facial tattoos. The database was originally collected by Ibsen et al in [4]. The images have all been aligned using the RetinaFace facial detector [52].…”

“…In this regard, facial tattoos are especially challenging, as they cause permanent alterations where ink is induced into the dermis layer of the skin. For instance, Ibsen et al investigated in [4] the impact of facial tattoos and paintings on state-of-the-art face recognition systems. The authors showed that tattoos might impair the recognition accuracy and thus the security of such a facial analysis system.…”