Feature Fusion Methods for Indexing and Retrieval of Biometric Data: Application to Face Recognition with Privacy Protection

Abstract

Computationally efficient, accurate, and privacypreserving data storage and retrieval are among the key challenges faced by practical deployments of biometric identification systems worldwide.In this work, a method of protected indexing of biometric data is presented. By utilising feature-level fusion of intelligently paired templates, a multi-stage search structure is created. During retrieval, the list of potential candidate identities is successively pre-filtered, thereby reducing the number of template comparisons necessary for a biometric identification transaction. Protection of the biometric probe templates, as well as the stored reference templates and the created index is carried out using homomorphic encryption.The proposed method is extensively evaluated in closedset and open-set identification scenarios on publicly available databases using two state-of-the-art open-source face recognition systems. With respect to a typical baseline algorithm utilising an exhaustive search-based retrieval algorithm, the proposed method enables a reduction of the computational workload associated with a biometric identification transaction by 90%, while simultaneously suffering no degradation of the biometric performance. Furthermore, by facilitating a seamless integration of template protection with open-source homomorphic encryption libraries, the proposed method guarantees unlinkability, irreversibility, and renewability of the protected biometric data.

Biometric Template Protection for Neural-Network-based Face Recognition Systems: A Survey of Methods and Evaluation Techniques

“…Current work in this area tends to focus on finding a balance between speeding up HE operations (e.g., by quantising/binarising the face templates or reducing their dimensionality) while simultaneously minimising losses in the resulting recognition accuracy (e.g., by trying to encrypt face templates in their original -usually floating-point -domain). Although [6]- [12] demonstrate important advances towards these goals for HE applied to both face verification (1-to-1 matching) [6], [7], [9] and identification (1-to-N matching) [8], [10]- [12], the greatest disadvantage of HE is that the encrypted templates remain secure only insofar as the corresponding decryption key remains secret; if an adversary gains access to this secret key, they can completely reverse the protection algorithm to obtain the original (unencrypted) template.…”

“…Non-NN [6] Accuracy (%) [7] TAR @ FAR = {0.01, 0.1, 1} (%) [8] DET (FNIR vs. FPIR) (%) ≈ [9] GAR @ FAR = {0.01, 0.1, 1} (%) [10] Rank-1 recognition rate (%), DET (FNIR vs. FPIR) (%) ≈ ≈ EER (%), FNIR @ FPIR = 0.1 (%) [11] Rank-1 accuracy (%) ≈…”

“…To complete the analysis on verbal implication renewability/unlinkability evaluations, we should mention the approach in [46] 10 . In this work, it was claimed that the lack of template storage means that the protected template cannot be compromised and cancellability is, therefore, ensured.…”