A Coupled Hidden Conditional Random Field Model for Simultaneous Face Clustering and Naming in Videos

Zhang, Yifan; Tang, Zhiqiang; Wu, Baoyuan; Ji, Qiang; Lu, Hanqing

doi:10.1109/tip.2016.2601491

Cited by 10 publications

(2 citation statements)

References 40 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…They then proposed a hybrid Bayesian network in [50] to capture label dependencies for AU recognition and AU intensity estimation. Temporal dynamics have been applied to model sequential data in different vision tasks [4], [28], [29], [53]- [55], [59], which can also be applied to AU intensity estimation. Probabilistic graphical models are efficient tools to capture temporal relationships among the AU intensities.…”

Section: Related Work a Shallow Models For Au Intensity Estimationmentioning

confidence: 99%

Semi-Supervised Deep Neural Network for Joint Intensity Estimation of Multiple Facial Action Units

et al. 2019

Self Cite

View full text Add to dashboard Cite

Facial action units (AUs) are defined to depict movements of facial muscles, which are basic elements to encode facial expressions. Automatic AU intensity estimation is an important task in affective computing. Previous works leverage the representation power of deep neural networks (DNNs) to improve the performance of intensity estimation. However, a large number of intensity annotations are required to train DNNs that contain millions of parameters. But it is expensive and difficult to build a largescale database with AU intensity annotation since AU annotation requires annotators have strong domain expertise. We propose a novel semi-supervised deep convolutional network that leverages extremely limited AU annotations for AU intensity estimation. It requires only intensity annotations of keyframes of training sequences. Domain knowledge on AUs is leveraged to provide weak supervisory information, including relative appearance similarity, temporal intensity ordering, facial symmetry, and contrastive appearance difference. We also propose a strategy to train a model for joint intensity estimation of multiple AUs under the setting of semi-supervised learning, which greatly improves the efficiency during inference. We perform empirical experiments on two public benchmark expression databases and make comparisons with state-ofthe-art methods to demonstrate the effectiveness of the proposed method.INDEX TERMS Facial action units, intensity estimation, deep learning, weakly supervised learning.

show abstract

Section: Related Work a Shallow Models For Au Intensity Estimationmentioning

confidence: 99%

Semi-Supervised Deep Neural Network for Joint Intensity Estimation of Multiple Facial Action Units

et al. 2019

Self Cite

View full text Add to dashboard Cite

show abstract

“…Started in 2011, the REPERE challenge aimed at supporting research on multimodal person recognition [20,26] to overcome some limitations of monomodal approaches, and annual evaluations were organized in 2012, 2013, and 2014. Much progress was achieved in either supervised or unsupervised multimodal person recognition [4,8,22,25,45,46,49,52,66,73,76].The MediaEval Person Discovery task [47] can be seen as a follow-up campaign, which focused on unsupervised person identification. In this challenge, participants were encouraged to develop multimodal approaches without using any prior biometric models.…”

Section: Introductionmentioning

confidence: 99%

Hierarchical multi-label propagation using speaking face graphs for multimodal person discovery

Fonseca

Sargent

Sicre

et al. 2020

Multimed Tools Appl

View full text Add to dashboard Cite

TV archives are growing in size so fast that manually indexing becomes unfeasible. Automatic indexing techniques can be applied to overcome this issue, and this work proposes an unsupervised technique for multimodal person discovery. To achieve this goal, we propose a hierarchical label propagation technique based on quasi-flat zones theory, that learns from labeled and unlabeled data and propagates names through a multimodal graph representation. In this representation, we combine audio, video, and text processing techniques to model the data as a graph of speaking faces. In the proposed modeling, we extract names via optical character recognition and propagate them through the graph using audiovisual relationships between speaking faces. We also use a random walk label propagation and two graph clustering strategies to serve as baselines. The proposed label propagation techniques always outperform the clustering baselines on the quantitative assessments. Our approach also outperforms all literature methods tested on the same dataset except for one, which uses a different preprocessing step. The proposed hierarchical label propagation and the random walk baseline produce highly equivalent results according to the Kappa coefficient, but the hierarchical propagation is parameter-free and over 9 times faster than the random walk under the same configurations.

show abstract

Self-supervised deep subspace clustering network for faces in videos

Qiu

Hao

2020

Vis Comput

View full text Add to dashboard Cite

A Coupled Hidden Conditional Random Field Model for Simultaneous Face Clustering and Naming in Videos

Cited by 10 publications

References 40 publications

Semi-Supervised Deep Neural Network for Joint Intensity Estimation of Multiple Facial Action Units

Semi-Supervised Deep Neural Network for Joint Intensity Estimation of Multiple Facial Action Units

Hierarchical multi-label propagation using speaking face graphs for multimodal person discovery

Self-supervised deep subspace clustering network for faces in videos

Contact Info

Product

Resources

About