Human pose regression by combining indirect part detection and contextual information

Luvizon, Diogo C.; Tabia, Hedi; Picard, David

doi:10.1016/j.cag.2019.09.002

Cited by 182 publications

(95 citation statements)

References 31 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The limitation of current regression methods is that the regression function is frequently sub-optimal. In order to tackle this weakness, the soft-argmax function [36] has been proposed to compute body joint coordinates from heat maps in a differentiable way.…”

Section: D Pose Estimationmentioning

confidence: 99%

“…For pose estimation, prediction blocks take as input the single frame features X p−1,l where Φ is the spatial softmax [36], and W p,l h and W p,l d are weight matrices. Probability maps and body joint depth maps encode, respectively, the probability of a body joint being at a given location and the depth with respect to the root joint, normalized in the interval [0, 1].…”

Section: Pose Regressionmentioning

confidence: 99%

“…Detection based approaches require the non-differentiable argmax function to recover the joint coordinates as a post processing stage, which breaks the backpropagation chain needed for end-toend learning. We propose to solve this problem by extending the differentiable soft-argmax [36,67] for joint 2D and 3D pose estimation. This allows us to stack action recognition on top of pose estimation, resulting in a multi-task framework trainable from end-to-end.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Multi-task Deep Learning for Real-Time 3D Human Pose Estimation and Action Recognition

Luvizon

Picard

Tabia

2020

IEEE Trans. Pattern Anal. Mach. Intell.

Self Cite

View full text Add to dashboard Cite

Human pose estimation and action recognition are related tasks since both problems are strongly dependent on the human body representation and analysis. Nonetheless, most recent methods in the literature handle the two problems separately. In this work, we propose a multi-task framework for jointly estimating 2D or 3D human poses from monocular color images and classifying human actions from video sequences. We show that a single architecture can be used to solve both problems in an efficient way and still achieves state-of-the-art or comparable results at each task while running with a throughput of more than 100 frames per second. The proposed method benefits from high parameters sharing between the two tasks by unifying still images and video clips processing in a single pipeline, allowing the model to be trained with data from different categories simultaneously and in a seamlessly way. Additionally, we provide important insights for end-to-end training the proposed multi-task model by decoupling key prediction parts, which consistently leads to better accuracy on both tasks. The reported results on four datasets (MPII, Human3.6M, Penn Action and NTU RGB+D) demonstrate the effectiveness of our method on the targeted tasks. Our source code and trained weights are publicly available at

show abstract

Section: D Pose Estimationmentioning

confidence: 99%

Section: Pose Regressionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Multi-task Deep Learning for Real-Time 3D Human Pose Estimation and Action Recognition

Luvizon

Picard

Tabia

2020

IEEE Trans. Pattern Anal. Mach. Intell.

Self Cite

View full text Add to dashboard Cite

show abstract

“…It's not difficult to understand that there exists an intrinsic connection between human actions and human poses (Wei et al 2016;Liu et al 2018;Cao et al 2017;Ning et al 2017;Luvizon et al 2017;Xiaohan Nie et al 2015). Different people may have different skin colors and appearance, dressing various clothes, but their poses are similar when they are doing the same action due to the homogeneity of human body.…”

Section: Combination Of Action Recognition and Pose Estimationmentioning

confidence: 99%

Turbo Learning Framework for Human-Object Interactions Recognition and Human Pose Estimation

Feng

Liu

et al. 2019

AAAI

View full text Add to dashboard Cite

Human-object interactions (HOI) recognition and pose estimation are two closely related tasks. Human pose is an essential cue for recognizing actions and localizing the interacted objects. Meanwhile, human action and their interacted objects' localizations provide guidance for pose estimation. In this paper, we propose a turbo learning framework to perform HOI recognition and pose estimation simultaneously. First, two modules are designed to enforce message passing between the tasks, i.e. pose aware HOI recognition module and HOI guided pose estimation module. Then, these two modules form a closed loop to utilize the complementary information iteratively, which can be trained in an end-to-end manner. The proposed method achieves the state-of-the-art performance on two public benchmarks including Verbs in COCO (V-COCO) and HICO-DET datasets.

show abstract

“…The intermediate supervision at each hourglass module benefits from previous module outputs, refining and improving final network predictions. Given its high performance, its conceptual simplicity, and that allows for an easy multitask integration among stacked modules, this architecture is serving as a baseline model in several works [30], [31], [32], [33], [34].…”

Section: A Multi-task Architecturementioning

confidence: 99%

Multi-task human analysis in still images: 2D/3D pose, depth map, and multi-part segmentation

Sánchez

Oliu

Madadi

et al. 2019

2019 14th IEEE International Conference on Automatic Face &Amp; Gesture Recognition (FG 2019)

View full text Add to dashboard Cite

While many individual tasks in the domain of human analysis have recently received an accuracy boost from deep learning approaches, multi-task learning has mostly been ignored due to a lack of data. New synthetic datasets are being released, filling this gap with synthetic generated data. In this work, we analyze four related human analysis tasks in still images in a multi-task scenario by leveraging such datasets. Specifically, we study the correlation of 2D/3D pose estimation, body part segmentation and full-body depth estimation. These tasks are learned via the well-known Stacked Hourglass module such that each of the task-specific streams shares information with the others. The main goal is to analyze how training together these four related tasks can benefit each individual task for a better generalization. Results on the newly released SURREAL dataset show that all four tasks benefit from the multi-task approach, but with different combinations of tasks: while combining all four tasks improves 2D pose estimation the most, 2D pose improves neither 3D pose nor full-body depth estimation. On the other hand 2D parts segmentation can benefit from 2D pose but not from 3D pose. In all cases, as expected, the maximum improvement is achieved on those human body parts that show more variability in terms of spatial distribution, appearance and shape, e.g. wrists and ankles.

show abstract

Human pose regression by combining indirect part detection and contextual information

Cited by 182 publications

References 31 publications

Multi-task Deep Learning for Real-Time 3D Human Pose Estimation and Action Recognition

Multi-task Deep Learning for Real-Time 3D Human Pose Estimation and Action Recognition

Turbo Learning Framework for Human-Object Interactions Recognition and Human Pose Estimation

Multi-task human analysis in still images: 2D/3D pose, depth map, and multi-part segmentation

Contact Info

Product

Resources

About