Video-based characters

Xufeng,; LiuYebin,; StollCarsten,; TompkinJames,; BharajGaurav,; DaiQionghai,; SeidelHans-Peter,; KautzJan,; Theobalt, Christian

doi:10.1145/2010324.1964927

Cited by 56 publications

(4 citation statements)

References 42 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The literature on human body modeling has a similar history to hands. The existing explicit mesh-based methods Xu et al 2011] rely on a template mesh obtained from a static scene and then learn appearance in the mesh space either by retrieval [Xu et al 2011] or by using a CNN to directly regress the texture map ]. The strong reliance on the template mesh has two limitations.…”

Section: Related Workmentioning

confidence: 99%

LiveHand: Real-time and Photorealistic Neural Hand Rendering

Mundra¹,

Mallikarjun²,

Wang³

et al. 2023

Preprint

View full text Add to dashboard Cite

Section: Related Workmentioning

confidence: 99%

LiveHand: Real-time and Photorealistic Neural Hand Rendering

Mundra¹,

Mallikarjun²,

Wang³

et al. 2023

Preprint

View full text Add to dashboard Cite

“…In light-field capture, for example, camera views with a relatively small baseline are interpolated to create novel views (Levoy and Hanrahan, 1996;Zitnick et al, 2004). Other work has used texture synthesis to warp 2D views into novel perspectives (Xu et al, 2011). In contrast to these techniques, FVV creates an animated 3D mesh of a character performance, which is reconstructed from a live-action performance using an array of inward-facing cameras arranged around the actor (Kanade et al, 1997).…”

Section: Filmed Content That Supports Viewing With Six Degrees Of Frementioning

confidence: 99%

Investigating the Perceived Strengths and Limitations of Free-Viewpoint Video

MacQuarrie

Steed

2020

Front. Virtual Real.

View full text Add to dashboard Cite

Free-viewpoint video (FVV) is a type of immersive content in which a character performance is filmed using an array of cameras and processed into a video-textured, animated 3D mesh. Although FVV content has a unique set of properties that differentiates it from other immersive media types, relatively little work explores the user experience of such content. As a preliminary investigation, we adopted an open-ended, qualitative approach to investigate these issues. Semi-structured interviews were conducted with six immersive content experts, exploring the perceived strengths and limitations of FVV as a content type. These interviews were analyzed using inductive thematic analysis. We identified five themes during our analysis: they don't look real, but that's okay; they can really move; they don't connect with me; encounter, legacy, and truth; no technology is an island. Our analysis reveals a wide range of future research directions and provides insight into which areas may produce the most benefit in relation to the user experience. We discuss, for example, the potential impact of difficulties in supporting user engagement, aspects related to visual quality such as the importance of responding realistically to environment lighting, and tensions between visual and behavioral quality. The analysis also highlights the complex interplay of factors related to the content itself, such as performance style and the use of creative production techniques to reduce the impact of potential limitations.

show abstract

“…Pixels instead of vertices. An alternative to 3D capture is video retrieval and warping (Xu et al 2011) where they look up video with the right motions and use an image-based approach. Jain et al (2010) fit an unclothed parametric body model to multi-camera and monocular image data.…”

Section: Related Workmentioning

confidence: 99%

ClothCap

Pons-Moll

Pujades

Hu³

et al. 2017

ACM Trans. Graph.

375

View full text Add to dashboard Cite

Fig. 1. ClothCap. From left to right: (1) An example 3D textured scan that is part of a 4D sequence. (2) Our multi-part aligned mesh model, layered over the body. (3) The estimated minimally clothed shape (MCS) under the clothing. (4) The body made fatter and dressed in the same clothing. Note that the clothing adapts in a natural way to the new body shape. (5) This new body shape posed in a new, never seen, pose. This illustrates how ClothCap supports a range of applications related to clothing capture, modeling, retargeting, reposing, and try-on.Designing and simulating realistic clothing is challenging. Previous methods addressing the capture of clothing from 3D scans have been limited to single garments and simple motions, lack detail, or require specialized texture patterns. Here we address the problem of capturing regular clothing on fully dressed people in motion. People typically wear multiple pieces of clothing at a time. To estimate the shape of such clothing, track it over time, and render it believably, each garment must be segmented from the others and the body. Our ClothCap approach uses a new multi-part 3D model of clothed bodies, automatically segments each piece of clothing, estimates the minimally clothed body shape and pose under the clothing, and tracks the 3D deformations of the clothing over time. We estimate the garments and their motion from 4D scans; that is, high-resolution 3D scans of the subject in motion at 60 fps. ClothCap is able to capture a clothed person in motion, extract their clothing, and retarget the clothing to new body shapes; this provides a step towards virtual try-on.

show abstract

Video-based characters

Cited by 56 publications

References 42 publications

LiveHand: Real-time and Photorealistic Neural Hand Rendering

LiveHand: Real-time and Photorealistic Neural Hand Rendering

Investigating the Perceived Strengths and Limitations of Free-Viewpoint Video

ClothCap

Contact Info

Product

Resources

About