Robust Facial Landmark Detection and Face Tracking in Thermal Infrared Images using Active Appearance Models

Kopaczka, Marcin; Acar, Kemal; Merhof, Dorit

doi:10.5220/0005716801500158

Cited by 30 publications

(28 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…With up to 10 fps, this method meets the runtime requirements for many real-time applications. Howevwer, the fitting performance needs to be analyzed critically since the authors of [17] have shown that these values usually result in poor fitting performance in infrared data.A deep alignment network trained by following the results given in [7]. With the trained algorithm, we implemented two different frame update strategies: an instance that is updated with the bounding box of the detected face (bounds-DAN) and a version that uses the detected landmark points directly for the shape update (shape-DAN).A ShapeNet following [18].…”

Section: Experiments and Resultsmentioning

confidence: 99%

“…Using the bounding box returned by the face detector and the model’s mean shape as initialization, we optimize the model parameters until optimal landmark positions are found. As shown in [17], feature-based AAMs allow precise landmark localization in thermal facial images. While yielding a high grade of precision, their main downside is the long fitting time as AAMs rely on multivariate optimization.…”

Section: Methodsmentioning

confidence: 99%

“…The database provides 2935 high-quality thermal face images with full manual annotations of 68 landmark points, providing for the first time a dataset of comparable size and quality to current face databases in the visual domain. The database has subsequently been used to train a set of facial landmark detection methods such as feature-based Active Appearance Models [17], A Deep Alignment Network [7] and a method for deep regression of shape parameters [18]. The quality and robustness of these methods clearly surpasses previous approaches, therefore, we will focus on utilizing these for our fully automatic face processing pipeline.…”

Section: State Of the Artmentioning

confidence: 99%

See 2 more Smart Citations

A Modular System for Detection, Tracking and Analysis of Human Faces in Thermal Infrared Recordings

Kopaczka

Breuer

Schock

et al. 2019

Sensors

Self Cite

View full text Add to dashboard Cite

We present a system that utilizes a range of image processing algorithms to allow fully automated thermal face analysis under both laboratory and real-world conditions. We implement methods for face detection, facial landmark detection, face frontalization and analysis, combining all of these into a fully automated workflow. The system is fully modular and allows implementing own additional algorithms for improved performance or specialized tasks. Our suggested pipeline contains a histogtam of oriented gradients support vector machine (HOG-SVM) based face detector and different landmark detecion methods implemented using feature-based active appearance models, deep alignment networks and a deep shape regression network. Face frontalization is achieved by utilizing piecewise affine transformations. For the final analysis, we present an emotion recognition system that utilizes HOG features and a random forest classifier and a respiratory rate analysis module that computes average temperatures from an automatically detected region of interest. Results show that our combined system achieves a performance which is comparable to current stand-alone state-of-the-art methods for thermal face and landmark datection and a classification accuracy of 65.75% for four basic emotions.

show abstract

Section: Experiments and Resultsmentioning

confidence: 99%

Section: Methodsmentioning

confidence: 99%

Section: State Of the Artmentioning

confidence: 99%

See 1 more Smart Citation

A Modular System for Detection, Tracking and Analysis of Human Faces in Thermal Infrared Recordings

Kopaczka

Breuer

Schock

et al. 2019

Sensors

Self Cite

View full text Add to dashboard Cite

show abstract

“…The automation of the measurement process may improve its use cases. In particular, recent advanced thermal ROI tracking algorithms could help this [9,[28][29][30]. However, this means that a thermal camera needs to be placed in front of the person during at least these measurements interfering with the activity and predicting the stress may indeed occur in that period of time.…”

Section: Introductionmentioning

confidence: 99%

Instant Automated Inference of Perceived Mental Stress through Smartphone PPG and Thermal Imaging

Cho

Julier

Bianchi-Berthouze

2018

Preprint

View full text Add to dashboard Cite

Background: A smartphone is a promising tool for daily cardiovascular measurement and mental stress monitoring. Photoplethysmography (PPG) and low---cost thermography can be used to create cheap, convenient and mobile systems. However, to achieve robustness, a person has to remain still for several minutes while a measurement is being taken. This is very cumbersome, and limits the usage in applications such producing instant measurements of stress. Objective: We propose to use smartphone---based mobile PPG and thermal imaging to provide a fast binary measure of stress responses to an event using dynamical physiological changes which occur within 20 seconds of the event finishing. Methods: We propose a system that uses a smartphone and its physiological sensors to reliably and continuously measure over a short window of time a person's blood volume pulse, the time interval between heartbeats (R---R interval) and the 1D thermal signature of the nose tip. 17 healthy participants, involved in a series of stress---inducing mental activities, measured their physiological response to stress in the 20 second---window immediately following each activity. A 10---cm Visual Analogue Scale was used by them to self---report their level of mental stress. As a main labeling strategy, normalized K---means clustering is used to better treat inter---personal differences in ratings. By taking an array of the R---R intervals and thermal directionality as a low---level feature input, we mainly use an artificial neural network to enable the automatic feature learning and the machine learning inference process. To compare the automated inference performance, we also extracted widely used high level features from HRV (e.g., LF/HF ratio) and the thermal signature and input them to a k---nearest neighbor to infer perceived stress levels.Results: First, we tested the physiological measurement reliability. The measured cardiac signals were considered highly reliable (signal goodness probability used, Mean=0.9584, SD=0.0151). The proposed 1D thermal signal processing algorithm effectively minimized the effect of respiratory cycles on detecting the apparent temperature of the nose tip (respiratory signal goodness probability Mean=0.8998 to Mean=0). Second, we tested the 20 seconds instant perceived stress inference performance. The best results were obtained by using automatic feature learning and classification using artificial neural networks rather than using pre---crafted features. The combination of both modalities produced higher accuracy on the

show abstract

“…However, these benchmark methods failed to detect landmarks correctly when applied to the IR images (Figure 3a and 3b). Previous work on infrared based facial analysis and ROI tracking primarily explored the use of standard machine learning techniques (Wesley, Buddharaju et al 2012, Ghiass, Arandjelović et al 2014, Kopaczka, Acar et al 2016, Kopaczka, Nestler et al 2017. These models allow optimal landmark detection in some cases but need further improvement as they rely on data attributes (features) which in the case of IR facial images lack the details present in visible spectrum images.…”

Section: Facial Landmark Estimationmentioning

confidence: 99%

Body and brain responses underlying salient emotional information processing

Sonkusare¹

View full text Add to dashboard Cite

Cognitive neuroscience has traditionally focused on simple tasks, presented sparsely and with abstract stimuli. While this approach has yielded fundamental insights into functional specialization in the brain, its ecological validity remains uncertain: Do these tasks fully capture how our brains function in the real world, where stimuli are dynamic, multimodal and crowded? Ecologically valid paradigms that approximate real life scenarios, using stimuli such as films, spoken narratives, music and multiperson games emerged in response to these concerns over a decade ago. We critically appraise whether this approach has delivered on its promise to deliver new insights into brain function. We highlight the challenges, technological innovations and clinical opportunities that are required should this field meet its full potential.

show abstract

Robust Facial Landmark Detection and Face Tracking in Thermal Infrared Images using Active Appearance Models

Cited by 30 publications

References 14 publications

A Modular System for Detection, Tracking and Analysis of Human Faces in Thermal Infrared Recordings

A Modular System for Detection, Tracking and Analysis of Human Faces in Thermal Infrared Recordings

Instant Automated Inference of Perceived Mental Stress through Smartphone PPG and Thermal Imaging

Body and brain responses underlying salient emotional information processing

Contact Info

Product

Resources

About