Improving cross-modal face recognition using polarimetric imaging

Short, Nathaniel J.; Hu, Shuowen; Gurram, Prudhvi; Gurton, Kristan P.; Chan, Anthony

doi:10.1364/ol.40.000882

Cited by 48 publications

(29 citation statements)

References 6 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Fig. 1: Examples of (a) visible-LWIR pair [47], (b) visible-polarimetric pair [54], (c) visible-MWIR pair [47], and (d) visible-NIR pair [47].…”

Section: Introductionmentioning

confidence: 99%

“…In particular, recent works have been proposed to use the polarization-state information of thermal emissions to enhance the performance of thermal face recognition [18], [49], [54], [76]. It has been shown that polarimetric-thermal images capture geometric and textural details of faces that are not present in the conventional thermal facial imagery [54]. As a result, the use of polarization-state information can improves cross-spectrum recognition performance over using intensity-only information from conventional thermal imagers.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Synthesis of High-Quality Visible Faces from Polarimetric Thermal Faces using Generative Adversarial Networks

Riggan

et al. 2019

Int J Comput Vis

Self Cite

View full text Add to dashboard Cite

The large domain discrepancy between faces captured in polarimetric (or conventional) thermal and visible domain makes cross-domain face verification a highly challenging problem for human examiners as well as computer vision algorithms. Previous approaches utilize either a twostep procedure (visible feature estimation and visible image reconstruction) or an input-level fusion technique, where different Stokes images are concatenated and used as a multichannel input to synthesize the visible image given the corresponding polarimetric signatures. Although these methods have yielded improvements, we argue that input-level fusion alone may not be sufficient to realize the full potential of the available Stokes images. We propose a Generative Adversarial Networks (GAN) based multi-stream featurelevel fusion technique to synthesize high-quality visible images from prolarimetric thermal images. The proposed network consists of a generator sub-network, constructed using an encoder-decoder network based on dense residual

show abstract

“…Fig. 1: Examples of (a) visible-LWIR pair [47], (b) visible-polarimetric pair [54], (c) visible-MWIR pair [47], and (d) visible-NIR pair [47].…”

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Synthesis of High-Quality Visible Faces from Polarimetric Thermal Faces using Generative Adversarial Networks

Riggan

et al. 2019

Int J Comput Vis

Self Cite

View full text Add to dashboard Cite

show abstract

“…It has been shown that polarimetric thermal imaging captures additional geometric and textural facial details compared to conventional thermal imaging [10]. Hence, the polarization-state information has been used to improve the performance of cross-spectrum face recognition [10,27,30,35,26,5]. A polarimetric, referred to as Stokes images, is composed of three channels: S0, S1 and S2.…”

Section: Introductionmentioning

confidence: 99%

“…The large domain discrepancy between these images makes the cross-spectrum matching problem very challenging. Various methods have been proposed in the literature for crossspectrum matching [10,27,30,35,26,16,21,18,2,28,24].…”

Section: Introductionmentioning

confidence: 99%

Polarimetric Thermal to Visible Face Verification via Self-Attention Guided Synthesis

Riggan

et al. 2019

2019 International Conference on Biometrics (ICB)

Self Cite

View full text Add to dashboard Cite

Polarimetric thermal to visible face verification entails matching two images that contain significant domain differences. Several recent approaches have attempted to synthesize visible faces from thermal images for cross-modal matching. In this paper, we take a different approach in which rather than focusing only on synthesizing visible faces from thermal faces, we also propose to synthesize thermal faces from visible faces. Our intuition is based on the fact that thermal images also contain some discriminative information about the person for verification. Deep features from a pre-trained Convolutional Neural Network (CNN) are extracted from the original as well as the synthesized images. These features are then fused to generate a template which is then used for verification. The proposed synthesis network is based on the self-attention generative adversarial network (SAGAN) which essentially allows efficient attention-guided image synthesis. Extensive experiments on the ARL polarimetric thermal face dataset demonstrate that the proposed method achieves state-of-the-art performance.

show abstract

“…The performance degradation is mainly due to the significant distributional change between the thermal and visible domains as well as a lack of sufficient data for training the deep networks for crossmodal matching. In many recent approaches, the polarization-state information of thermal emissions has been used to achieve improved cross-spectrum face recognition performance [9,27,31,36,26] since it captures geometric and textural details of faces that are not present in the conventional thermal facial images [31,9]. A polarimetric thermal image consists of four Stokes images: S 0 , S 1 , S 2 , and degree-of-linearpolarization (DoLP), where S 0 indicates the conventional total intensity thermal image, S 1 captures the horizontal and vertical polarization-state information, S 2 captures the diagonal polarization-state information and DoLP describes the portion of an electromagnetic wave that is linearly polarized [9].…”

Section: Introductionmentioning

confidence: 99%

Polarimetric Thermal to Visible Face Verification via Attribute Preserved Synthesis

Patel

2018

2018 IEEE 9th International Conference on Biometrics Theory, Applications and Systems (BTAS)

View full text Add to dashboard Cite

Thermal to visible face verification is a challenging problem due to the large domain discrepancy between the modalities. Existing approaches either attempt to synthesize visible faces from thermal faces or extract robust features from these modalities for cross-modal matching. In this paper, we take a different approach in which we make use of the attributes extracted from the visible image to synthesize the attribute-preserved visible image from the input thermal image for cross-modal matching. A pre-trained VGG-Face network is used to extract the attributes from the visible image. Then, a novel Attribute Preserved Generative Adversarial Network (AP-GAN) is proposed to synthesize the visible image from the thermal image guided by the extracted attributes. Finally, a deep network is used to extract features from the synthesized image and the input visible image for verification. Extensive experiments on the ARL Polarimetric face dataset show that the proposed method achieves significant improvements over the state-of-the-art methods.

show abstract

Improving cross-modal face recognition using polarimetric imaging

Cited by 48 publications

References 6 publications

Synthesis of High-Quality Visible Faces from Polarimetric Thermal Faces using Generative Adversarial Networks

Synthesis of High-Quality Visible Faces from Polarimetric Thermal Faces using Generative Adversarial Networks

Polarimetric Thermal to Visible Face Verification via Self-Attention Guided Synthesis

Polarimetric Thermal to Visible Face Verification via Attribute Preserved Synthesis

Contact Info

Product

Resources

About