Generative adversarial network-based synthesis of visible faces from polarimetrie thermal faces

Abstract

The large domain discrepancy between faces captured in polarimetric (or conventional) thermal and visible domain makes cross-domain face recognition quite a challenging problem for both human-examiners and computer vision algorithms. Previous approaches utilize a two-step procedure (visible feature estimation and visible image reconstruction) to synthesize the visible image given the corresponding polarimetric thermal image. However, these are regarded as two disjoint steps and hence may hinder the performance of visible face reconstruction. We argue that joint optimization would be a better way to reconstruct more photo-realistic images for both computer vision algorithms and human-examiners to examine. To this end, this paper proposes a Generative Adversarial Network-based Visible Face Synthesis (GAN-VFS) method to synthesize more photo-realistic visible face images from their corresponding polarimetric images. To ensure that the encoded visible-features contain more semantically meaningful information in reconstructing the visible face image, a guidance sub-network is involved into the training procedure. To achieve photo realistic property while preserving discriminative characteristics for the reconstructed outputs, an identity loss combined with the perceptual loss are optimized in the framework. Multiple experiments evaluated on different experimental protocols demonstrate that the proposed method achieves state-of-the-art performance.

Polarimetric Thermal to Visible Face Verification via Self-Attention Guided Synthesis

“…Regarding Protocol I, we evaluate and compare the performance of the proposed method with recent state-of-theart methods [35,19,27,26,5,36]. Figure 5 shows the evaluation performance for two different experimental settings, S0 (representing conventional thermal) and Polar separately.…”

“…It has been shown that polarimetric thermal imaging captures additional geometric and textural facial details compared to conventional thermal imaging [10]. Hence, the polarization-state information has been used to improve the performance of cross-spectrum face recognition [10,27,30,35,26,5]. A polarimetric, referred to as Stokes images, is composed of three channels: S0, S1 and S2.…”

“…The large domain discrepancy between these images makes the cross-spectrum matching problem very challenging. Various methods have been proposed in the literature for crossspectrum matching [10,27,30,35,26,16,21,18,2,28,24].…”

“…Various synthesis-based methods have also been proposed in the literature [26,35,38,27] to perform crossmodal mapping at the image level for direct use in existing visible-based matchers. Riggan et al [27] trained a regression network to estimate the mapping between features from both visible and thermal then reconstruct the visible face based on the estimated features.…”

“…Riggan et al [27] trained a regression network to estimate the mapping between features from both visible and thermal then reconstruct the visible face based on the estimated features. Zhang et al [35,36] leveraged generative adversarial networks (GANs) to synthesize visible images from polarimetric thermal images. Riggan et al [26] proposed a global and local region-based synthesis network to transform the thermal image into the visible spectrum.…”