Semi- and weakly-supervised human pose estimation

Abstract

For human pose estimation in still images, this paper proposes three semi-and weakly-supervised learning schemes. While recent advances of convolutional neural networks improve human pose estimation using supervised training data, our focus is to explore the semi-and weakly-supervised schemes. Our proposed schemes initially learn conventional model(s) for pose estimation from a small amount of standard training images with human pose annotations. For the first semi-supervised learning scheme, this conventional pose model detects candidate poses in training images with no human annotation. From these candidate poses, only true-positives are selected by a classifier using a pose feature representing the configuration of all body parts. The accuracies of these candidate pose estimation and true-positive pose selection are improved by action labels provided to these images in our second and third learning schemes, which are semi-and weakly-supervised learning. While the first and second learning schemes select only poses that are similar to those in the supervised training data, the third scheme selects more true-positive poses that are significantly different from any supervised poses. This pose selection is achieved by pose clustering using outlier pose detection with Dirichlet process mixtures and the Bayes factor. The proposed schemes are validated with large-scale human pose datasets.

Deep Graph Pose: a semi-supervised deep graphical model for improved animal pose tracking

“…Semi-supervised learning aims to fully utilize unlabeled or weaklylabeled data to gain additional insights into the structure of the data [41,42,43]. Many pose estimation algorithms have adopted such learning schemes to enhance the performance given limited training data [44,35]. One conceptually similar "weakly-supervised" approach is described in [45], who trained a network to extract flying objects (obeying Newtonian acceleration) simply by constraining the output to resemble a parabola.…”

MirrorNet: A Deep Reflective Approach to 2D Pose Estimation for Single-Person Images

“…An optimal objective function was proposed for evaluating the relations between pairs of joints [4], [9], [26], [43]. Recently, some studies have assessed the correctness of inferred poses using additional networks [6], [13], [29] or compensated for the lack of data samples with data augmentation [34], [46]. Here, we review plausibilityaware methods of pose estimation and fidelity-aware methods of image processing.…”

“…To compensate for a lack of training data, Ukita and Uematsu [46] took a semi-and weakly-supervised approach that uses non-annotated images and action labels of images (e.g., baseball and volleyball) to estimate the poses of humans from a part of paired data. Peng et al [34] proposed an efficient data augmentation method that generates hard-to-recognize images with adversarial training.…”

“…Second, the performance of the supervised approach is limited by the amount of paired pose-image data. To overcome this limitation, data augmentation techniques [34] and the use of metadata [46] and non-annotated data [12], [46] have been proposed. A unified solution to these complementary problems, however, remains an open question.…”

Vision skeleton trajectory based motion assessment system for healthcare rehabilitation

“…These objects are variable, including not only brightness and perspective, but also the deformation of the object. Ukita and Uematsu [13] used part‐segment features for estimating an articulated pose in still images. With advance of deep learning methods in computer vision, many methods have proposed effectively pose and skeleton estimation algorithm based on deep learning theory.…”