End-to-End Recovery of Human Shape and Pose

Kanazawa, Angjoo; Black, Michael J.; Jacobs, David W.; Malik, Jitendra

doi:10.1109/cvpr.2018.00744

Cited by 1,730 publications

(2,067 citation statements)

References 33 publications

Supporting

Mentioning

2,056

Contrasting

Unclassified

Order By: Relevance

“…However, because they require each character to be tracked by a bounding box, they only reconstruct single‐person skeletons at a time, making them unsuitable for closely interacting characters. More recently, an enormous effort has been devoted to deep and convolutional methods that map all human pixels of an RGB image to 3D surface of the human body …”

Section: Related Workmentioning

confidence: 99%

Real‐time 3D human pose and motion reconstruction from monocular RGB videos

Yiannakides

Aristidou

Chrysanthou

2019

Computer Animation & Virtual

View full text Add to dashboard Cite

Real‐time three‐dimensional (3D) pose estimation is of high interest in interactive applications, virtual reality, activity recognition, and most importantly, in the growing gaming industry. In this work, we present a method that captures and reconstructs the 3D skeletal pose and motion articulation of multiple characters using a monocular RGB camera. Our method deals with this challenging, but useful, task by taking advantage of the recent development in deep learning that allows two‐dimensional (2D) pose estimation of multiple characters and the increasing availability of motion capture data. We fit 2D estimated poses, extracted from a single camera via OpenPose, with a 2D multiview joint projections database that is associated with their 3D motion representations. We then retrieve the 3D body pose of the tracked character, ensuring throughout that the reconstructed movements are natural, satisfy the model constraints, are within a feasible set, and are temporally smooth without jitters. We demonstrate the performance of our method in several examples, including human locomotion, simultaneously capturing of multiple characters, and motion reconstruction from different camera views.

show abstract

Section: Related Workmentioning

confidence: 99%

Real‐time 3D human pose and motion reconstruction from monocular RGB videos

Yiannakides

Aristidou

Chrysanthou

2019

Computer Animation & Virtual

View full text Add to dashboard Cite

show abstract

“…Several of newer approaches in motion capture exploit machine learning techniques to infer pose information from sensor input [BS10, YY14, YSD*16]. The recent advancements in deep learning motivated the use of artificial neural networks for motion capture purposes [BKL*16, PIT*16, RLGK16, CSWS17, MSS*17, EdAJ*17, KBJM18].…”

Section: Related Workmentioning

confidence: 99%

Markerless Multiview Motion Capture with 3D Shape Model Adaptation

Fechteler

Hilsmann

Eisert

2019

Computer Graphics Forum

View full text Add to dashboard Cite

In this paper, we address simultaneous markerless motion and shape capture from 3D input meshes of partial views onto a moving subject. We exploit a computer graphics model based on kinematic skinning as template tracking model. This template model consists of vertices, joints and skinning weights learned a priori from registered full‐body scans, representing true human shape and kinematics‐based shape deformations. Two data‐driven priors are used together with a set of constraints and cues for setting up sufficient correspondences. A Gaussian mixture model‐based pose prior of successive joint configurations is learned to soft‐constrain the attainable pose space to plausible human poses. To make the shape adaptation robust to outliers and non‐visible surface regions and to guide the shape adaptation towards realistically appearing human shapes, we use a mesh‐Laplacian‐based shape prior. Both priors are learned/extracted from the training set of the template model learning phase. The output is a model adapted to the captured subject with respect to shape and kinematic skeleton as well as the animation parameters to resemble the observed movements. With example applications, we demonstrate the benefit of such footage. Experimental evaluations on publicly available datasets show the achieved natural appearance and accuracy.

show abstract

“…Recently, Peng et al [PKM * 18] successfully extracted reference motion from videos. They used pose estimation techniques from images [WRKS16,KBJM18]. This inspired our study, where we use figure skating videos on YouTube to obtain key pose data.…”

Section: Related Workmentioning

confidence: 99%

“…When input images from video are given by the user, the first step is a data acquisition process. To obtain the reference data for skating simulation, we extracted key poses from inputs using human mesh recovery (HMR) [KBJM18], a pose estimation technique, which is explained in Section 5. Secondly, We used trajectory optimization to complete the reference motion proposed by Al Borno et al [ABDLH13].…”

Section: Overviewmentioning

confidence: 99%

See 1 more Smart Citation

Figure Skating Simulation from Video

Park

Lee

2019

Computer Graphics Forum

View full text Add to dashboard Cite

Figure skating is one of the most popular ice sports at the Winter Olympic Games. The skaters perform several skating skills to express the beauty of the art on ice. Skating involves moving on ice while wearing skate shoes with thin blades; thus, it requires much practice to skate without losing balance. Moreover, figure skating presents dynamic moves, such as jumping, artistically. Therefore, demonstrating figure skating skills is even more difficult to achieve than basic skating, and professional skaters often fall during Winter Olympic performances. We propose a system to demonstrate figure skating motions with a physically simulated human‐like character. We simulate skating motions with non‐holonomic constraints, which make the skate blade glide on the ice surface. It is difficult to obtain reference motions from figure skaters because figure skating motions are very fast and dynamic. Instead of using motion capture data, we use key poses extracted from videos on YouTube and complete reference motions using trajectory optimization. We demonstrate figure skating skills, such as crossover, three‐turn, and even jump. Finally, we use deep reinforcement learning to generate a robust controller for figure skating skills.

show abstract

End-to-End Recovery of Human Shape and Pose

Cited by 1,730 publications

References 33 publications

Real‐time 3D human pose and motion reconstruction from monocular RGB videos

Real‐time 3D human pose and motion reconstruction from monocular RGB videos

Markerless Multiview Motion Capture with 3D Shape Model Adaptation

Figure Skating Simulation from Video

Contact Info

Product

Resources

About