From Scores to Face Templates: A Model-Based Approach

Abstract

Regeneration of templates from match scores has security and privacy implications related to any biometric authentication system. We propose a novel paradigm to reconstruct face templates from match scores using a linear approach. It proceeds by first modeling the behavior of the given face recognition algorithm by an affine transformation. The goal of the modeling is to approximate the distances computed by a face recognition algorithm between two faces by distances between points, representing these faces, in an affine space. Given this space, templates from an independent image set (break-in) are matched only once with the enrolled template of the targeted subject and match scores are recorded. These scores are then used to embed the targeted subject in the approximating affine (non-orthogonal) space. Given the coordinates of the targeted subject in the affine space, the original template of the targeted subject is reconstructed using the inverse of the affine transformation. We demonstrate our ideas using three, fundamentally different, face recognition algorithms: Principal Component Analysis (PCA) with Mahalanobis cosine distance measure, Bayesian intra-extrapersonal classifier (BIC), and a feature-based commercial algorithm. To demonstrate the independence of the break-in set with the gallery set, we select face templates from two different databases: Face Recognition Grand Challenge (FRGC) and Facial Recognition Technology (FERET) Database (FERET). With an operational point set at 1 percent False Acceptance Rate (FAR) and 99 percent True Acceptance Rate (TAR) for 1,196 enrollments (FERET gallery), we show that at most 600 attempts (score computations) are required to achieve a 73 percent chance of breaking in as a randomly chosen target subject for the commercial face recognition system. With similar operational set up, we achieve a 72 percent and 100 percent chance of breaking in for the Bayesian and PCA based face recognition systems, respectively. With three different levels of score quantization, we achieve 69 percent, 68 percent and 49 percent probability of break-in, indicating the robustness of our proposed scheme to score quantization. We also show that the proposed reconstruction scheme has 47 percent more probability of breaking in as a randomly chosen target subject for the commercial system as compared to a hill climbing approach with the same number of attempts. Given that the proposed template reconstruction method uses distinct face templates to reconstruct faces, this work exposes a more severe form of vulnerability than a hill climbing kind of attack where incrementally different versions of the same face are used. Also, the ability of the proposed approach to reconstruct actual face templates of the users increases privacy concerns in biometric systems.

Reconstructing faces from their signatures using RBF regression

“…Most of the face reconstruction methods have been experimented in this context (e.g. [1,2,21]), explaining why they observed that simple gradient based optimization allows to reconstruct faces easily in their cases.…”

“…However, as pointed out by [21], this hill-climbing optimization is rather inefficient as it involves the optimization of a non-convex criterion and requires a large number of attempts before succeeding. Consequently, [21] proposed to use a different paradigm.…”

Globally Linear Embedding of Biometric Scores: An Empirical Study

“…The stress on linearity is not just based on appeal to Occam's razor, but also because of possible impact on template security, privacy, indexing. In [11] we have shown that if we can build globally linear models of the distances, then it is possible to construct approximating linear models that map biometric templates to distances, as computed by a recognition algorithm. Based on this linear model, it is possible to reconstruct any target template given just the similarity scores of this target template with a set of break-in query templates.…”

Hill-climbing attack to an Eigenface-based face verification system

“…In [2] a model-based attack which is capable of reconstructing the user's face images from the matching scores is presented. The method has the strong constraint of needing a large number of real face images to initialize the algorithm.…”

“…Most of the existing works studying the vulnerabilities of biometric systems to attacks against the inner modules of the system, apart from [2], use some type of variant of the hillclimbing algorithm presented in [3]. Some examples include an attack to a face-based system in [4], and to a standard and a Match-on-Card minutiae-based fingerprint verification systems in [5] and [6] respectively.…”