FERA 2017 - Addressing Head Pose in the Third Facial Expression Recognition and Analysis Challenge

Valstar, Michel; Sánchez-Lozano, Enrique; Cohn, Jeffrey F.; Jeni, László A.; Girard, Jeffrey M.; Zhang, Zheng; Liu, Yin; Pantić, Maja

doi:10.1109/fg.2017.107

Cited by 100 publications

(69 citation statements)

References 36 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Subsets of these data have been used in FERA 2017 [29] and 3DFAW [21]. The CNN contains three convolutional layers and two fully connected layers (see Figure 1).…”

Section: Methodsmentioning

confidence: 99%

See 1 more Smart Citation

Automated Affect Detection in Deep Brain Stimulation for Obsessive-Compulsive Disorder

Cohn

Jeni

Ertuğrul

et al. 2018

Proceedings of the 20th ACM International Conference on Multimodal Interaction

Self Cite

View full text Add to dashboard Cite

Automated measurement of affective behavior in psychopathology has been limited primarily to screening and diagnosis. While useful, clinicians more often are concerned with whether patients are improving in response to treatment. Are symptoms abating, is affect becoming more positive, are unanticipated side effects emerging? When treatment includes neural implants, need for objective, repeatable biometrics tied to neurophysiology becomes especially pressing. We used automated face analysis to assess treatment response to deep brain stimulation (DBS) in two patients with intractable obsessive-compulsive disorder (OCD). One was assessed intraoperatively following implantation and activation of the DBS device. The other was assessed three months post-implantation. Both were assessed during DBS on and o conditions. Positive and negative valence were quantified using a CNN trained on normative data of 160 non-OCD participants. Thus, a secondary goal was domain transfer of the classifiers. In both contexts, DBS-on resulted in marked positive affect. In response to DBS-off, affect flattened in both contexts and alternated with increased negative affect in the outpatient setting. Mean AUC for domain transfer was 0.87. These findings suggest that parametric variation of DBS is strongly related to affective behavior and may introduce vulnerability for negative affect in the event that DBS is discontinued.

show abstract

“…Subsets of these data have been used in FERA 2017 [29] and 3DFAW [21]. The CNN contains three convolutional layers and two fully connected layers (see Figure 1).…”

Section: Methodsmentioning

confidence: 99%

“…Automated emotion recognition from facial expression is an active area of research [26, 29]. In clinical contexts, investigators have detected occurrence of depression, autism, conflict, and PTSD from visual features (i.e., face and body expression or movement) [7, 10, 18, 22, 25, 27].…”

Section: Introductionmentioning

confidence: 99%

Automated Affect Detection in Deep Brain Stimulation for Obsessive-Compulsive Disorder

Cohn

Jeni

Ertuğrul

et al. 2018

Proceedings of the 20th ACM International Conference on Multimodal Interaction

Self Cite

View full text Add to dashboard Cite

show abstract

“…Creating publicly available "in-the-wild" dataset is therefore of importance. Occurrence performance is measured in terms of F1, and intensity in terms of ICC (see [181] for details).…”

Section: Challenges and Opportunitiesmentioning

confidence: 99%

“…Occurrence detection Intensity estimation Amirian et al [8] -0.295 Batista et al [18] 0.506 0.399 He et al [73] 0.507 -Li et al [95] 0.495 -Tang et al [166] 0.574 -Zhou et al [218] -0.445 Baseline [181] 0.452 0.217…”

Section: Teammentioning

confidence: 99%

Automatic Analysis of Facial Actions: A Survey

Martínez

Valstar

Jiang

et al. 2019

IEEE Trans. Affective Comput.

Self Cite

238

153

View full text Add to dashboard Cite

Abstract-As one of the most comprehensive and objective ways to describe facial expressions, the Facial Action Coding System (FACS) has recently received significant attention. Over the past 30 years, extensive research has been conducted by psychologists and neuroscientists on various aspects of facial expression analysis using FACS. Automating FACS coding would make this research faster and more widely applicable, opening up new avenues to understanding how we communicate through facial expressions. Such an automated process can also potentially increase the reliability, precision and temporal resolution of coding. This paper provides a comprehensive survey of research into machine analysis of facial actions. We systematically review all components of such systems: pre-processing, feature extraction and machine coding of facial actions. In addition, the existing FACS-coded facial expression databases are summarised. Finally, challenges that have to be addressed to make automatic facial action analysis applicable in real-life situations are extensively discussed. There are two underlying motivations for us to write this survey paper: the first is to provide an up-to-date review of the existing literature, and the second is to offer some insights into the future of machine recognition of facial actions: what are the challenges and opportunities that researchers in the field face.

show abstract

“…For each sequence, three types of meta data are provided, including 27 manually annotated AUs, automatically tracked head pose (pitch, yaw, and roll), and 83 2D/3D facial landmarks. It is noted that the dataset in the Facial Expression Recognition and Analysis (FERA) 2017 challenge [Valstar et al 2017] was derived from the 3D model of the BP4D database. The dataset comprises of 2,952 videos for training, 1,431 videos for validation and 1,080 videos for test.…”

Section: )mentioning

confidence: 99%

Facial Expression Analysis under Partial Occlusion

et al. 2018

View full text Add to dashboard Cite

Automatic machine-based Facial Expression Analysis (FEA) has made substantial progress in the past few decades driven by its importance for applications in psychology, security, health, entertainment and human computer interaction. The vast majority of completed FEA studies are based on non-occluded faces collected in a controlled laboratory environment. Automatic expression recognition tolerant to partial occlusion remains less understood, particularly in real-world scenarios. In recent years, efforts investigating techniques to handle partial occlusion for FEA have seen an increase. The context is right for a comprehensive perspective of these developments and the state of the art from this perspective. This survey provides such a comprehensive review of recent advances in dataset creation, algorithm development, and investigations of the effects of occlusion critical for robust performance in FEA systems. It outlines existing challenges in overcoming partial occlusion and discusses possible opportunities in advancing the technology. To the best of our knowledge, it is the first FEA survey dedicated to occlusion and aimed at promoting better informed and benchmarked future work.

show abstract

FERA 2017 - Addressing Head Pose in the Third Facial Expression Recognition and Analysis Challenge

Cited by 100 publications

References 36 publications

Automated Affect Detection in Deep Brain Stimulation for Obsessive-Compulsive Disorder

Automated Affect Detection in Deep Brain Stimulation for Obsessive-Compulsive Disorder

Automatic Analysis of Facial Actions: A Survey

Facial Expression Analysis under Partial Occlusion

Contact Info

Product

Resources

About