Personalizing Human Video Pose Estimation

Abstract

We propose a personalized ConvNet pose estimator that automatically adapts itself to the uniqueness of a person's appearance to improve pose estimation in long videos.We make the following contributions: (i) we show that given a few high-precision pose annotations, e.g. from a generic ConvNet pose estimator, additional annotations can be generated throughout the video using a combination of image-based matching for temporally distant frames, and dense optical flow for temporally local frames; (ii) we develop an occlusion aware self-evaluation model that is able to automatically select the high-quality and reject the erroneous additional annotations; and (iii) we demonstrate that these high-quality annotations can be used to fine-tune a ConvNet pose estimator and thereby personalize it to lock on to key discriminative features of the person's appearance. The outcome is a substantial improvement in the pose estimates for the target video using the personalized ConvNet compared to the original generic ConvNet.Our method outperforms the state of the art (including top ConvNet methods) by a large margin on three standard benchmarks, as well as on a new challenging YouTube video dataset. Furthermore, we show that training from the automatically generated annotations can be used to improve the performance of a generic ConvNet on other benchmarks.

Adversarial Video Compression Guided by Soft Edge Detection

“…(1) from one video of the YouTube Pose dataset [1]. The next four frames in Figure 4 (a) are outputs from the soft edge detector for several different levels of quantization.…”

“…YouTube Pose dataset [1]: This dataset is comprised of 45 YouTube videos. Each video contains between 2,000 and 20,000 frames, covering a broad range of activities by a single person: dancing, stand-up comedy, sports and so on.…”

“…Each video contains between 2,000 and 20,000 frames, covering a broad range of activities by a single person: dancing, stand-up comedy, sports and so on. We excluded 13 of the videos owing to their long durations 1…”

PoseTrack: Joint Multi-person Pose Estimation and Tracking

“…The MPII Multi-Person Pose [1] 40,522 We Are Family [11] 3131 MPII Multi-Person Pose [30] 14,161 MS-COCO Keypoints [25] 105,698 J-HMDB [21] 32,173 Penn-Action [45] 159,633 VideoPose [35] 1286 Poses-in-the-wild [9] 831 YouTube Pose [7] 5000 FYDP [36] 1680 UYDP [36] 2000…”

“…Many applications, such as mentioned before, however, aim to analyze human body motion over time. While there exists a notable number of works that track the pose of a single person in a video [28,9,44,33,46,20,29,7,13,18], multi-person human pose estimation in unconstrained videos has not been addressed in the literature. In this work, we address the problem of tracking the poses of multiple persons in an unconstrained setting.…”