MultiPoseNet: Fast Multi-Person Pose Estimation Using Pose Residual Network

Kocabas, Muhammed; Karagoz, Salih; Akbaş, Emre

doi:10.1007/978-3-030-01252-6_26

Cited by 276 publications

(140 citation statements)

References 57 publications

Supporting

Mentioning

140

Contrasting

Order By: Relevance

“…Papandreou et al [49] proposed to detect individual keypoints and predict their relative displacements, allowing a greedy decoding process to group keypoints into person instances. Kocabas et al [50] proposed a Pose Residual Network which receives keypoint and person detections, and then assigns keypoints to detected person bounding boxes. Nie et al [51] proposed to partition all keypoint detections using dense regressions from keypoint candidates to centroids of persons in the image.…”

Section: Multi-person Pose Estimationmentioning

confidence: 99%

OpenPose: Realtime Multi-Person 2D Pose Estimation Using Part Affinity Fields

Cao

Hidalgo

Simon

et al. 2021

IEEE Trans. Pattern Anal. Mach. Intell.

3,497

2,155

View full text Add to dashboard Cite

Realtime multi-person 2D pose estimation is a key component in enabling machines to have an understanding of people in images and videos. In this work, we present a realtime approach to detect the 2D pose of multiple people in an image. The proposed method uses a nonparametric representation, which we refer to as Part Affinity Fields (PAFs), to learn to associate body parts with individuals in the image. This bottom-up system achieves high accuracy and realtime performance, regardless of the number of people in the image. In previous work, PAFs and body part location estimation were refined simultaneously across training stages. We demonstrate that a PAF-only refinement rather than both PAF and body part location refinement results in a substantial increase in both runtime performance and accuracy. We also present the first combined body and foot keypoint detector, based on an internal annotated foot dataset that we have publicly released. We show that the combined detector not only reduces the inference time compared to running them sequentially, but also maintains the accuracy of each component individually. This work has culminated in the release of OpenPose, the first open-source realtime system for multi-person 2D pose detection, including body, foot, hand, and facial keypoints.Index Terms-2D human pose estimation, 2D foot keypoint estimation, real-time, multiple person, part affinity fields.

show abstract

Section: Multi-person Pose Estimationmentioning

confidence: 99%

OpenPose: Realtime Multi-Person 2D Pose Estimation Using Part Affinity Fields

Cao

Hidalgo

Simon

et al. 2021

IEEE Trans. Pattern Anal. Mach. Intell.

3,497

2,155

View full text Add to dashboard Cite

show abstract

“…Single-person pose estimation [41,34,42,30,17] localizes 2D body keypoints of a person in a cropped image. There are two categories of multi-person pose estimation methods: top-down methods [10,17,15,13] that first detect people in the image and then apply single-person pose estimation to the cropped image of each person, and bottom-up methods [25,29,8,35,18] that first detect all keypoints and then group them into different people. In general, the top-down methods are more accurate, while the bottomup methods are relatively faster.…”

Section: Single-view Pose Estimationmentioning

confidence: 99%

Fast and Robust Multi-Person 3D Pose Estimation From Multiple Views

Dong

Wen

Huang

et al. 2019

2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

187

176

View full text Add to dashboard Cite

This paper addresses the problem of 3D pose estimation for multiple people in a few calibrated camera views. The main challenge of this problem is to find the cross-view correspondences among noisy and incomplete 2D pose predictions. Most previous methods address this challenge by directly reasoning in 3D using a pictorial structure model, which is inefficient due to the huge state space. We propose a fast and robust approach to solve this problem. Our key idea is to use a multi-way matching algorithm to cluster the detected 2D poses in all views. Each resulting cluster encodes 2D poses of the same person across different views and consistent correspondences across the keypoints, from which the 3D pose of each person can be effectively inferred. The proposed convex optimization based multi-way matching algorithm is efficient and robust against missing and false detections, without knowing the number of people in the scene. Moreover, we propose to combine geometric and appearance cues for cross-view matching. The proposed approach achieves significant performance gains from the state-of-the-art (96.3% vs. 90.6% and 96.9% vs. 88% on the Campus and Shelf datasets, respectively), while being efficient for real-time applications.

show abstract

“…Human pose estimation. Multi-person pose estimation has attracted substantial attention over the last years [8,14,15,19,23,28,32,33]. One of the most popular datasets is the MPII multi-person pose estimation dataset [3].…”

Section: Related Workmentioning

confidence: 99%

Learning to Train with Synthetic Humans

Hoffmann

Tzionas

Black

et al. 2019

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Neural networks need big annotated datasets for training. However, manual annotation can be too expensive or even unfeasible for certain tasks, like multi-person 2D pose estimation with severe occlusions. A remedy for this is synthetic data with perfect ground truth. Here we explore two variations of synthetic data for this challenging problem; a dataset with purely synthetic humans and a real dataset augmented with synthetic humans. We then study which approach better generalizes to real data, as well as the influence of virtual humans in the training loss. Using the augmented dataset, without considering synthetic humans in the loss, leads to the best results. We observe that not all synthetic samples are equally informative for training, while the informative samples are different for each training stage. To exploit this observation, we employ an adversarial student-teacher framework; the teacher improves the student by providing the hardest samples for its current state as a challenge. Experiments show that the student-teacher framework outperforms normal training on the purely synthetic dataset.

show abstract

MultiPoseNet: Fast Multi-Person Pose Estimation Using Pose Residual Network

Cited by 276 publications

References 57 publications

OpenPose: Realtime Multi-Person 2D Pose Estimation Using Part Affinity Fields

OpenPose: Realtime Multi-Person 2D Pose Estimation Using Part Affinity Fields

Fast and Robust Multi-Person 3D Pose Estimation From Multiple Views

Learning to Train with Synthetic Humans

Contact Info

Product

Resources

About