Rethinking Keypoint Representations: Modeling Keypoints and Poses as Objects for Multi-Person Human Pose Estimation

McNally, William; Vats, Kanav; Wong, Alexander; McPhee, John

doi:10.48550/arxiv.2111.08557

Cited by 4 publications

(3 citation statements)

References 43 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…KAPAO is an efficient single-stage multi-person human pose estimation method that models keypoints and poses as objects within a dense anchor-based detection framework. KAPAO simultaneously detects pose objects and keypoint objects and fuses the detections to predict human poses [28].…”

Section: Kapao-based Pose Estimationmentioning

confidence: 99%

BabyPose: Real-Time Decoding of Baby’s Non-Verbal Communication Using 2D Video-Based Pose Estimation

Yurtsever

Eken

2022

IEEE Sensors J.

View full text Add to dashboard Cite

People use various communication methods to interact with each other, such as speech and body movements. However, communication methods in babies differ according to their developmental periods. Babies may not be able to express themselves verbally, but they do have their own communication methods. Naturally, it becomes important for parents to understand these signs/poses. Human pose estimation is extensively employed in different applications such as video surveillance, sports analysis and medical support/aid. The goal of this research is to assist new parents by addressing pose-based real-time body movements of babies (such as arching back, head banging, kicking legs, rubbing eyes, stretching, sucking fingers) and making sense of their activities. This is the first study to estimate poses on babies. In this paper, a baby pose dataset is created from 156 video clips through online video sharing platforms. Key-points are obtained from three different pose estimators -OpenPose, AlphaPose, and KAPAO. Different LSTM models are used to recognize the babies' activities and different performance metrics are used to compare these models. The best model has 99% accuracy and 0.0712 loss ratio. Also, babies are tracked in real-time via DeepSORT algorithm. Experimental results show that the proposed system is very promising and filling a gap in making sense of baby poses and monitoring them.

show abstract

Section: Kapao-based Pose Estimationmentioning

confidence: 99%

BabyPose: Real-Time Decoding of Baby’s Non-Verbal Communication Using 2D Video-Based Pose Estimation

Yurtsever

Eken

2022

IEEE Sensors J.

View full text Add to dashboard Cite

show abstract

“…At present, a lot of excellent works [2,[9][10][11][12][13][14][15] based on the Transformer [8] or its variants [16][17][18][19][20] have emerged. There are also many excellent works [21][22][23][24][25][26] based on Transformer in the field of human posture estimation. PRTR [21] utilizes the encoder-decoder structure of transformers to perform a regression-based person and keypoint detection.…”

Section: Vision Transformer For Human Pose Estimationmentioning

confidence: 99%

RefinePose: Towards More Refined Human Pose Estimation

et al. 2022

View full text Add to dashboard Cite

Human pose estimation is a very important research topic in computer vision and attracts more and more researchers. Recently, ViTPose based on heatmap representation refreshed the state of the art for pose estimation methods. However, we find that ViTPose still has room for improvement in our experiments. On the one hand, the PatchEmbedding module of ViTPose uses a convolutional layer with a stride of 14 × 14 to downsample the input image, resulting in the loss of a significant amount of feature information. On the other hand, the two decoding methods (Classical Decoder and Simple Decoder) used by ViTPose are not refined enough: transpose convolution in the Classical Decoder produces the inherent chessboard effect; the upsampling factor in the Simple Decoder is too large, resulting in the blurry heatmap. To this end, we propose a novel pose estimation method based on ViTPose, termed RefinePose. In RefinePose, we design the GradualEmbedding module and Fusion Decoder, respectively, to solve the above problems. More specifically, the GradualEmbedding module only downsamples the image to 1/2 of the original size in each downsampling stage, and it reduces the input image to a fixed size (16 × 112 in ViTPose) through multiple downsampling stages. At the same time, we fuse the outputs of max pooling layers and convolutional layers in each downsampling stage, which retains more meaningful feature information. In the decoding stage, the Fusion Decoder designed by us combines bilinear interpolation with max unpooling layers, and gradually upsamples the feature maps to restore the predicted heatmap. In addition, we also design the FeatureAggregation module to aggregate features after sampling (upsampling and downsampling). We validate the RefinePose on the COCO dataset. The experiments show that RefinePose has achieved better performance than ViTPose.

show abstract

“…We also introduce a variant, BiFenceNet, that outperforms JLJA while using the same 2D skeleton data. This way, coaches and analysts could extract information directly from videos, by training FenceNet on 2D pose data extracted from an offthe-shelf 2D pose estimator [6,32,51], as seen in Fig. 1.…”

Section: Introductionmentioning

confidence: 99%

FenceNet: Fine-grained Footwork Recognition in Fencing

Zhu¹,

Wong²,

McPhee³

2022

Preprint

Self Cite

View full text Add to dashboard Cite

Current data analysis for the Canadian Olympic fencing team is primarily done manually by coaches and analysts. Due to the highly repetitive, yet dynamic and subtle movements in fencing, manual data analysis can be inefficient and inaccurate. We propose FenceNet as a novel architecture to automate the classification of fine-grained footwork techniques in fencing. FenceNet takes 2D pose data as input and classifies actions using a skeleton-based action recognition approach that incorporates temporal convolutional networks to capture temporal information. We train and evaluate FenceNet on the Fencing Footwork Dataset (FFD), which contains 10 fencers performing 6 different footwork actions for 10-11 repetitions each (652 total videos). FenceNet achieves 85.4% accuracy under 10-fold cross-validation, where each fencer is left out as the test set. This accuracy is within 1% of the current state-of-theart method, JLJA (86.3%), which selects and fuses features engineered from skeleton data, depth videos, and inertial measurement units. BiFenceNet, a variant of FenceNet that captures the "bidirectionality" of human movement through two separate networks, achieves 87.6% accuracy, outperforming JLJA. Since neither FenceNet nor BiFenceNet requires data from wearable sensors, unlike JLJA, they could be directly applied to most fencing videos, using 2D pose data as input extracted from off-the-shelf 2D human pose estimators. In comparison to JLJA, our methods are also simpler as they do not require manual feature engineering, selection, or fusion.

show abstract

Rethinking Keypoint Representations: Modeling Keypoints and Poses as Objects for Multi-Person Human Pose Estimation

Cited by 4 publications

References 43 publications

BabyPose: Real-Time Decoding of Baby’s Non-Verbal Communication Using 2D Video-Based Pose Estimation

BabyPose: Real-Time Decoding of Baby’s Non-Verbal Communication Using 2D Video-Based Pose Estimation

RefinePose: Towards More Refined Human Pose Estimation

FenceNet: Fine-grained Footwork Recognition in Fencing

Contact Info

Product

Resources

About