A Thermal Infrared Face Database With Facial Landmarks and Emotion Labels

Kopaczka, Marcin; Kolk, Raphael; Schock, Justus; Burkhard, Felix; Merhof, Dorit

doi:10.1109/tim.2018.2884364

Cited by 67 publications

(61 citation statements)

References 41 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…All algorithms operate exclusively on thermal image data without the need for an additional visual or 3D depth camera. Wherever a machine learning algorithm was part of our algorithm, we used the annotated database introduced in [7] for training. The database contains 2935 manually annotated grayscale thermal images acquired with an InfraTec HD Head 820 thermal camera with a spatial resolution of 1024 × 768 pixels and a thermal resolution of 0.03K and using the acquisition software provided by the camera manufacturer.…”

Section: Methodsmentioning

confidence: 99%

“…To assess wether current landmark detection methods that are based on deep learning can be used in our system, we used the database to train a Deep Alignment Network (DAN) [8] for precise landmark detection. It has already been shown in [7] that DANs can be used for robust landmark detection in thermal images and yield extremly precise results. Therefore, adding this method allowed incorporating a very current and advanced method from the visual domain into our pipeline.…”

Section: Methodsmentioning

confidence: 99%

“…In their research, these authors were able to show that face detectors based on machine learning constantly outperform specialized approaches if a suitable training database is provided. With increasing computing power and several recently available thermal face databases [7,8], the current trend in thermal face detection is the use of learning-based detctors.…”

Section: State Of the Artmentioning

confidence: 99%

“…The database provides 2935 high-quality thermal face images with full manual annotations of 68 landmark points, providing for the first time a dataset of comparable size and quality to current face databases in the visual domain. The database has subsequently been used to train a set of facial landmark detection methods such as feature-based Active Appearance Models [17], A Deep Alignment Network [7] and a method for deep regression of shape parameters [18]. The quality and robustness of these methods clearly surpasses previous approaches, therefore, we will focus on utilizing these for our fully automatic face processing pipeline.…”

Section: State Of the Artmentioning

confidence: 99%

See 3 more Smart Citations

A Modular System for Detection, Tracking and Analysis of Human Faces in Thermal Infrared Recordings

Kopaczka

Breuer

Schock

et al. 2019

Sensors

Self Cite

View full text Add to dashboard Cite

We present a system that utilizes a range of image processing algorithms to allow fully automated thermal face analysis under both laboratory and real-world conditions. We implement methods for face detection, facial landmark detection, face frontalization and analysis, combining all of these into a fully automated workflow. The system is fully modular and allows implementing own additional algorithms for improved performance or specialized tasks. Our suggested pipeline contains a histogtam of oriented gradients support vector machine (HOG-SVM) based face detector and different landmark detecion methods implemented using feature-based active appearance models, deep alignment networks and a deep shape regression network. Face frontalization is achieved by utilizing piecewise affine transformations. For the final analysis, we present an emotion recognition system that utilizes HOG features and a random forest classifier and a respiratory rate analysis module that computes average temperatures from an automatically detected region of interest. Results show that our combined system achieves a performance which is comparable to current stand-alone state-of-the-art methods for thermal face and landmark datection and a classification accuracy of 65.75% for four basic emotions.

show abstract

Section: Methodsmentioning

confidence: 99%

Section: Methodsmentioning

confidence: 99%

Section: State Of the Artmentioning

confidence: 99%

Section: State Of the Artmentioning

confidence: 99%

See 2 more Smart Citations

A Modular System for Detection, Tracking and Analysis of Human Faces in Thermal Infrared Recordings

Kopaczka

Breuer

Schock

et al. 2019

Sensors

Self Cite

View full text Add to dashboard Cite

show abstract

“…This algorithm requires many labeled images. However, these image processing methods are currently unavailable in the thermal domain [14]. Compared to RGB images, the algorithms used for thermal image processing still lack robustness and accuracy.…”

Section: Introductionmentioning

confidence: 99%

A Novel Facial Thermal Feature Extraction Method for Non-Contact Healthcare System

et al. 2020

View full text Add to dashboard Cite

The non-contact healthcare system is a system that can avoid germ infection, and can also provide comfortable and convenient health care services for the caregivers. In the current thermal imaging research, local or small area images are used to represent the overall temperature information of the participants, but the temperature feature of the face should not only be used in a small part and ignore other parts. The facial thermal image can show clear temperature feature but is not conducive to meaningful feature extraction. Therefore, this research proposed a novel facial thermal image feature extraction method, which is used facial landmarks to detect and cut 12 blocks to establish a new feature matrix based on color mean values and standard deviation values. It can establish clear features on facial thermal images. The core part of the proposed healthcare system is the use of a deep learning framework, which is based on CAFFE under the DIGITS platform. The CAFFE runs the classic CNN, GoogLeNet. Based on the acquired images and new feature types, four models were trained, which were used for the raw RGB image, raw thermal image, RGB feature image, and thermal feature image. In the experiment, 800 images were used for training and validation, and 200 images were used for testing. An additional 40 images were used for random testing. The experimental results show that RGB images cannot be effectively used, thermal images can effectively predict the health status, and thermal feature images have the highest prediction accuracy. INDEX TERMS Non-contact healthcare system, thermal feature extraction, deep learning, GoogLeNet.

show abstract

Multimodal data acquisition at SARS‐CoV‐2 drive through screening centers: Setup description and experiences in Saarland, Germany

Flotho

Bhamborae

Grün

et al. 2021

Journal of Biophotonics

View full text Add to dashboard Cite

SARS‐CoV‐2 drive through screening centers (DTSC) have been implemented worldwide as a fast and secure way of mass screening. We use DTSCs as a platform for the acquisition of multimodal datasets that are needed for the development of remote screening methods. Our acquisition setup consists of an array of thermal, infrared and RGB cameras as well as microphones and we apply methods from computer vision and computer audition for the contactless estimation of physiological parameters. We have recorded a multimodal dataset of DTSC participants in Germany for the development of remote screening methods and symptom identification. Acquisition in the early stages of a pandemic and in regions with high infection rates can facilitate and speed up the identification of infection specific symptoms and large‐scale data acquisition at DTSC is possible without disturbing the flow of operation.

show abstract

A Thermal Infrared Face Database With Facial Landmarks and Emotion Labels

Cited by 67 publications

References 41 publications

A Modular System for Detection, Tracking and Analysis of Human Faces in Thermal Infrared Recordings

A Modular System for Detection, Tracking and Analysis of Human Faces in Thermal Infrared Recordings

A Novel Facial Thermal Feature Extraction Method for Non-Contact Healthcare System

Multimodal data acquisition at SARS‐CoV‐2 drive through screening centers: Setup description and experiences in Saarland, Germany

Contact Info

Product

Resources

About