Cross-Spectrum Thermal Face Pattern Generator

Abstract

Conversion of a visible face image into a thermal face image (V2T), or one thermal face image into another one given a different target temperature (T2T), is required in applications such as thermography, human body thermal pattern analysis, and surveillance using cross-spectral imaging. In this work, we propose to use conditional generative adversarial networks (cGAN) with cGAN loss, perceptual loss, and temperature loss to solve the conversion tasks. In our experiment, we used Carl and SpeakingFaces Databases. Frèchet Inception Distance (FID) is used to evaluate the generated images. As well, face recognition was applied to assess the performance of our models. For the V2T task, the FID of the generated thermal images reached a low value of 57.3. For the T2T task, we achieved a rank-1 face recognition rate of 91.0% which indicates that the generated thermal images preserve the majority of the identity information.INDEX TERMS Generative adversarial networks, image-to-image translation, thermal pattern generation, face recognition, biometrics.

Thermal–Visible Face Recognition Based on CNN Features and Triple Triplet Configuration for On-the-Move Identity Verification

“…Most of the works listed in the related works section are focused on the identification task, and thus are not directly comparable with our undertaking. The work of Chen et al [6], focused on the verification task, presents a TAR of 51.24% for 1% of FAR, while in [15], the GAN-based method tested on the Speaking Faces database achieved a Rank-1 of 78%.…”

“…The work of Cao et al [15] presents a conversion of a visible face image into a thermal face image (V2T) and a thermal face image into another one with a different temperature of the face (T2T). They developed a framework based on a U-Net generator and a six-layer PatchGAN discriminator.…”

When Visible-to-Thermal Facial GAN Beats Conditional Diffusion

“…The metadata can be affixed with variables such as Planck constants and emissivity where temperature values are directly provided, or can be calculated using thermodynamic equations from software. For example, Cao et al, [17] minimize a temperature loss using a "temperature matrix" provided as metadata in the form of .bmp files from the Carl Database [44] through a pix2pix cGAN. Their temperature vector is composed of a single scalar value duplicated across the matrix representing only the forehead temperature, whereas ThermalGAN's vector consists of undisclosed background and object temperatures.…”

“…2) CycleGAN [28], 3) ThermalGAN [16], and 4) favtGAN [13]). These baselines were selected based on their use in existing VT/TV studies such as [7,17,36,42]. We combine the Eurecom and Devcom datasets together in order to train the favtgan [13] model that requires two datasets captured from similar thermal sensors.…”

“…For facial VT translation, paired methods are preferable, in order to preserve the physiological mapping which is considered a biometric [13,14,15]. Yet, few works have explored how to estimate temperature from thermogram pixels where some works use metadata supplied by the camera, that demonstrate high quality results [16,17]. More recently, works using diffusion models, formally known as Denoising Diffusion Probabilistic Models (DDPM) [18,19], have enabled an alternate framework to generate diverse, high resolution art.…”