Towards Real-Time Head Pose Estimation: Exploring Parameter-Reduced Residual Networks on In-the-wild Datasets

Rieger, Ines; Hauenstein, Thomas; Hettenkofer, Sebastian; Garbas, Jens-Uwe

doi:10.1007/978-3-030-22999-3_12

Cited by 8 publications

(6 citation statements)

References 27 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For example, Hui et al [34] proposed a very compact LiteFlowNet which is 30 times smaller in the model size and 1.36 times faster in the running speed in comparison with the state-of-the-art CNNs for optical flow estimation. In [35], Rieger et al explored parameter-reduced residual networks on in-the-wild datasets, targeting real-time head pose estimation. They experimented with various ResNet architectures with a varying number of layers to handle different image sizes (including low-resolution images).…”

Section: Methodsmentioning

confidence: 99%

Cost-Effective CNNs for Real-Time Micro-Expression Recognition

et al. 2020

View full text Add to dashboard Cite

Micro-Expression (ME) recognition is a hot topic in computer vision as it presents a gateway to capture and understand daily human emotions. It is nonetheless a challenging problem due to ME typically being transient (lasting less than 200 ms) and subtle. Recent advances in machine learning enable new and effective methods to be adopted for solving diverse computer vision tasks. In particular, the use of deep learning techniques on large datasets outperforms classical approaches based on classical machine learning which rely on hand-crafted features. Even though available datasets for spontaneous ME are scarce and much smaller, using off-the-shelf Convolutional Neural Networks (CNNs) still demonstrates satisfactory classification results. However, these networks are intense in terms of memory consumption and computational resources. This poses great challenges when deploying CNN-based solutions in many applications, such as driver monitoring and comprehension recognition in virtual classrooms, which demand fast and accurate recognition. As these networks were initially designed for tasks of different domains, they are over-parameterized and need to be optimized for ME recognition. In this paper, we propose a new network based on the well-known ResNet18 which we optimized for ME classification in two ways. Firstly, we reduced the depth of the network by removing residual layers. Secondly, we introduced a more compact representation of optical flow used as input to the network. We present extensive experiments and demonstrate that the proposed network obtains accuracies comparable to the state-of-the-art methods while significantly reducing the necessary memory space. Our best classification accuracy was 60.17% on the challenging composite dataset containing five objectives classes. Our method takes only 24.6 ms for classifying a ME video clip (less than the occurrence time of the shortest ME which lasts 40 ms). Our CNN design is suitable for real-time embedded applications with limited memory and computing resources.

show abstract

Section: Methodsmentioning

confidence: 99%

Cost-Effective CNNs for Real-Time Micro-Expression Recognition

et al. 2020

View full text Add to dashboard Cite

show abstract

“…Our backbone is a parameter-reduced 18-layer ResNet (ResNet18) [20] based on He et al [6]. Small changes were made in the output layer which uses a sigmoid function and in the activation functions, which are ReLU units.…”

Section: B Trainingmentioning

confidence: 99%

Multi-Label Class Balancing Algorithm for Action Unit Detection

Pahl¹,

Rieger²,

Seuß³

2020

Preprint

Self Cite

View full text Add to dashboard Cite

Isolated facial movements, so-called Action Units, can describe combined emotions or physical states such as pain. As datasets are limited and mostly imbalanced, we present an approach incorporating a multi-label class balancing algorithm. This submission is subject to the Action Unit detection task of the Affective Behavior Analysis in-the-wild (ABAW) challenge at the IEEE Conference on Face and Gesture Recognition.

show abstract

“…Our backbone is a parameter-reduced 18-layer ResNet with input images of size 112x112 pixels (ResNet18-112) [25] (Fig. 1b).…”

Section: Trainingmentioning

confidence: 99%

Unique Class Group Based Multi-Label Balancing Optimizer for Action Unit Detection

Rieger

Pahl

Seuß

2020

2020 15th IEEE International Conference on Automatic Face and Gesture Recognition (FG 2020)

Self Cite

View full text Add to dashboard Cite

Balancing methods for single-label data cannot be applied to multi-label problems as they would also resample the samples with high occurrences. We propose to reformulate this problem as an optimization problem in order to balance multi-label data. We apply this balancing algorithm to training datasets for detecting isolated facial movements, socalled Action Units. Several Action Units can describe combined emotions or physical states such as pain. As datasets in this area are limited and mostly imbalanced, we show how optimized balancing and then augmentation can improve Action Unit detection. At the IEEE Conference on Face and Gesture Recognition 2020, we ranked third in the Affective Behavior Analysis in-the-wild (ABAW) challenge for the Action Unit detection task.

show abstract

Towards Real-Time Head Pose Estimation: Exploring Parameter-Reduced Residual Networks on In-the-wild Datasets

Cited by 8 publications

References 27 publications

Cost-Effective CNNs for Real-Time Micro-Expression Recognition

Cost-Effective CNNs for Real-Time Micro-Expression Recognition

Multi-Label Class Balancing Algorithm for Action Unit Detection

Unique Class Group Based Multi-Label Balancing Optimizer for Action Unit Detection

Contact Info

Product

Resources

About