A Large-Scale, Time-Synchronized Visible and Thermal Face Dataset

Poster, Domenick; Thielke, Matthew; Nguyen, Robert; Rajaraman, Srinivasan; Di, Xing; Fondje, Cedric Nimpa; Patel, Vishal M.; Short, Nathaniel J.; Riggan, Benjamin S.; Nasrabadi, Nasser M.; Hu, Shuowen

doi:10.1109/wacv48630.2021.00160

Cited by 39 publications

(21 citation statements)

References 30 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…ARL-VTF dataset: In [5], authors made available the DEVCOM Army Research Laboratory Visible-Thermal Face Dataset (ARL-VTF). The dataset contains heterogeneous data from 395 subjects with three visible spectra as well as one thermal (long-wave infrared-LWIR) camera, with over 500,000 images altogether.…”

Section: A Databases and Protocolsmentioning

confidence: 99%

Section: A Databases and Protocolsmentioning

confidence: 99%

“…The arXiv:2210.06529v1 [cs.CV] 12 Oct 2022 performance of the networks trained using RGB images deteriorates when used with images captured with other sensing modalities [4]. Further, there are not many largescale heterogeneous datasets to train models specifically for HFR [5]. Leveraging pre-trained FR models, which are trained using large-scale face recognition datasets, appears to be a reasonable strategy to follow in this limited data domain.…”

Section: Introductionmentioning

confidence: 99%

“…Leveraging pre-trained FR models, which are trained using large-scale face recognition datasets, appears to be a reasonable strategy to follow in this limited data domain. Furthermore, special emphasis should be put on developing approaches that work with a limited amount of paired data as this type of heterogeneous data is costly to acquire and not readily available [5].…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Prepended Domain Transformer: Heterogeneous Face Recognition Without Bells and Whistles

George

Mohammadi

Marcel

2023

IEEE Trans.Inform.Forensic Secur.

View full text Add to dashboard Cite

Heterogeneous Face Recognition (HFR) refers to matching face images captured in different domains, such as thermal to visible images (VIS), sketches to visible images, near-infrared to visible, and so on. This is particularly useful in matching visible spectrum images to images captured from other modalities. Though highly useful, HFR is challenging because of the domain gap between the source and target domain. Often, large-scale paired heterogeneous face image datasets are absent, preventing training models specifically for the heterogeneous task. In this work, we propose a surprisingly simple, yet, very effective method for matching face images across different sensing modalities. The core idea of the proposed approach is to add a novel neural network block called Prepended Domain Transformer (PDT) in front of a pre-trained face recognition (FR) model to address the domain gap. Retraining this new block with few paired samples in a contrastive learning setup was enough to achieve stateof-the-art performance in many HFR benchmarks. The PDT blocks can be retrained for several source-target combinations using the proposed general framework. The proposed approach is architecture agnostic, meaning they can be added to any pre-trained FR models. Further, the approach is modular and the new block can be trained with a minimal set of paired samples, making it much easier for practical deployment. The source code and protocols will be made available publicly.

show abstract

Section: A Databases and Protocolsmentioning

confidence: 99%

Section: A Databases and Protocolsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Prepended Domain Transformer: Heterogeneous Face Recognition Without Bells and Whistles

George

Mohammadi

Marcel

2023

IEEE Trans.Inform.Forensic Secur.

View full text Add to dashboard Cite

show abstract

“…However, the bounding boxes can be obtained using the coordinates of facial landmarks. The ARL-VTF dataset [21] contains more than 500,000 thermal-visual image pairs of 395 individuals. The dataset provides bounding boxes and six point landmarks.…”

Section: Related Workmentioning

confidence: 99%

TFW: Annotated Thermal Faces in the Wild Dataset

Kuzdeuov¹,

Aubakirova²,

Koishigarina³

et al. 2022

Preprint

View full text Add to dashboard Cite

Face detection and subsequent localization of facial landmarks are the primary steps in many face applications. Numerous algorithms and benchmark datasets have been introduced to develop robust models for the visible domain. However, varying conditions of illumination still pose challenging problems. In this regard, thermal cameras are employed to address this problem because they operate on longer wavelengths. However, thermal face and facial landmark detection in the wild is an open research problem because most of the existing thermal datasets were collected in controlled environments. In addition, many of them were not annotated with face bounding boxes and facial landmarks. In this work, we present a thermal face dataset with manually labeled bounding boxes and facial landmarks to address these problems. The dataset contains 9,982 images of 147 subjects, collected under controlled and uncontrolled conditions. As a baseline, we trained YOLOv5 object detection model and its adaptation for face detection, YOLO5Face, on our dataset. In addition to our test set, we evaluated the models on the external RWTH-Aachen thermal face dataset to show the efficacy of our dataset. We have made the dataset, source code, and pre-trained models publicly available at https://github.com/IS2AI/TFW to bolster research in thermal face analysis.<br>

show abstract