Fusion of IR and visible light modalities for face recognition

Abstract

We present a low resolution face recognition technique based on a special type of convolutional neural network which is trained to extract facial features from face images and project them onto a low-dimensional space. The network is trained to reconstruct a reference image chosen beforehand, and it has been applied in visible and infrared light. Since the learning phase is achieved separately for the two modalities, the projections, and then the new spaces, are uncorrelated for the two networks. However, by normalizing the results of these two non-linear approaches, we can merge them according to a measure of saliency computed dynamically. We experimentally show that our approach obtain good results in terms of precision and robustness, especially on new and unseen subjects.

IR and visible face identification via sparse representation

“…In both experiments, there is only one image per subject in the gallery, acting like a 1image-to-enroll scenario. The Same-session experiment is composed of: Tables I and II. The results for the Same-session experiment, which is an easy test, are quite the same as those given in [4] based on a Convolutional Neural Network, or those in [5] using PCA. However, there is a significant improvement of recognition rates for the Time-lapse experiment.…”

“…Results at section IV-C show that visible modality performs better than IR. This result has already been shown in [4] and [5]. However the sets of mismatched probes of the two classifiers do not necessarily overlap.…”

“…They show that the fusion scheme always improves the best result of one modality alone. They are also always better than those given in [4] and [5].…”

“…Since the classifiers for the two modalities yield decision rankings as results, we chose to merge the results on the decision level. We have tested some algorithms like the one presented in [4], which realizes a weighted sum of the scores of the two modalities according to a measure of saliency computed dynamically. Nevertheless, we found that the simple sum rule on the residual gives the best results.…”

Genetic programming for multibiometrics

“…Buyssens et al [23] showed the interest of biometric fusion for face recognition combining the image in visible and infrared color spaces with convolutional neural networks. In [24], Mantalvao and Freire have combined keystroke dynamics with voice recognition, it seems it is the first time that multibiometrics has been done with keystroke dynamics and another biometric modality.…”