Just‐in‐time, viable, 3‐D avatars from scans

Feng, Andrew; Rosenberg, Evan Suma; Shapiro, Ari

doi:10.1002/cav.1769

Cited by 20 publications

(16 citation statements)

References 31 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…They significantly vary in terms of the degree of achievable realism, the technical complexities, and finally, the overall reconstruction costs involved. So far, rather complex and expensive multi-camera rigs achieve the highest quality by using high-quality image sensors, e.g., as described by Feng et al (2017) or Achenbach et al (2017). However, approaches for reconstructing virtual humans from input data produced by more affordable consumer hardware, e.g., single 2D images (Alldieck et al, 2019a) or smartphone videos (Ichim et al, 2015;Wenninger et al, 2020), become more popular and elaborate.…”

Section: Introductionmentioning

confidence: 99%

Affordable But Not Cheap: A Case Study of the Effects of Two 3D-Reconstruction Methods of Virtual Humans

Bartl

Wenninger

Wolf

et al. 2021

Front. Virtual Real.

View full text Add to dashboard Cite

Realistic and lifelike 3D-reconstruction of virtual humans has various exciting and important use cases. Our and others’ appearances have notable effects on ourselves and our interaction partners in virtual environments, e.g., on acceptance, preference, trust, believability, behavior (the Proteus effect), and more. Today, multiple approaches for the 3D-reconstruction of virtual humans exist. They significantly vary in terms of the degree of achievable realism, the technical complexities, and finally, the overall reconstruction costs involved. This article compares two 3D-reconstruction approaches with very different hardware requirements. The high-cost solution uses a typical complex and elaborated camera rig consisting of 94 digital single-lens reflex (DSLR) cameras. The recently developed low-cost solution uses a smartphone camera to create videos that capture multiple views of a person. Both methods use photogrammetric reconstruction and template fitting with the same template model and differ in their adaptation to the method-specific input material. Each method generates high-quality virtual humans ready to be processed, animated, and rendered by standard XR simulation and game engines such as Unreal or Unity. We compare the results of the two 3D-reconstruction methods in an immersive virtual environment against each other in a user study. Our results indicate that the virtual humans from the low-cost approach are perceived similarly to those from the high-cost approach regarding the perceived similarity to the original, human-likeness, beauty, and uncanniness, despite significant differences in the objectively measured quality. The perceived feeling of change of the own body was higher for the low-cost virtual humans. Quality differences were perceived more strongly for one’s own body than for other virtual humans.

show abstract

Section: Introductionmentioning

confidence: 99%

Affordable But Not Cheap: A Case Study of the Effects of Two 3D-Reconstruction Methods of Virtual Humans

Bartl

Wenninger

Wolf

et al. 2021

Front. Virtual Real.

View full text Add to dashboard Cite

show abstract

“…Passive‐vision methods acquire 3D information from 2D images that are usually captured from multi‐view viewpoints simultaneously in a studio environment. There are many passive multi‐view systems [LDX10, Rem04, FRS17] focused on human body shape reconstruction. For the acquisition systems, sparse camera setups [SH07, VPB∗09, TNM09] with extremely wide baselines or full‐body capture systems [VPB∗09, LDX10, JLT∗15] with low percentages of body pixels limit the quality of reconstructed human body shapes.…”

Section: Related Workmentioning

confidence: 99%

“…To further propel the development of passive methods in the field of human body shape reconstruction, we propose a passive multi-view system utilizing a robust camera calibration ap- proach and an advanced PatchMatch MVS method to acquire accurate results. Although our body scanning pipeline is similar to the previous work [FRS17], they focused on the efficiency of model generation while the proposed system focuses on the accuracy.…”

Section: Related Workmentioning

confidence: 99%

A Robust Multi‐View System for High‐Fidelity Human Body Shape Reconstruction

Zhang

Wang

Liu

et al. 2021

Computer Graphics Forum

View full text Add to dashboard Cite

This paper proposes a passive multi‐view system for human body shape reconstruction, namely RHF‐Human, to overcome several challenges including accurate calibration and stereo matching in self‐occluded and low‐texture skin regions. The reconstruction process includes four steps: capture, multi‐view camera calibration, dense reconstruction, and meshing. The capture system, which consists of 90 digital single‐lens reflex cameras, is single‐shot to avoid nonrigid deformation of the human body. Two technical contributions are made: (1) a two‐step robust multi‐view calibration approach that improves calibration accuracy and saves calibration time for each new human body acquired and (2) an accurate PatchMatch multi‐view stereo method for dense reconstruction to perform correct matching in self‐occluded and low‐texture skin regions and to reduce the noise caused by body hair. Experiments on models of various genders, poses, and skin with different amounts of body hair show the robustness of the proposed system. A high‐fidelity human body shape dataset with 227 models is constructed, and the average accuracy is within 1.5 mm. The system provides a new scheme for the accurate reconstruction of nonrigid human models based on passive vision and has good potential in fashion design and health care.

show abstract

“…Researchers from the University of Southern California have been experimenting with cruder but faster variations of body and face multicamera photogrammetry to generate just-in-time avatars; their project focuses on creating viable 3-D gamer avatars that can be incorporated into a game or real-time simulation without requiring extensive predevelopment time; the team has been able to cut down the creation time to less than 20 min (Feng, Rosenberg, & Shapiro, 2017). While such fast turnaround solutions are still a long way from becoming viable pipelines for producing key player characters, they could enable gamers to project their own likenesses into their favorite games, as well as enhancing communication- and interaction-based games and simulations.…”

Section: Challenges and Future Trendsmentioning

confidence: 99%

Use of Photogrammetry in Video Games: A Historical Overview

Statham

2018

Games and Culture

View full text Add to dashboard Cite

In 2014, the developers of The Vanishing of Ethan Carter revealed that the stunning settings in the game were created using photogrammetry, a modeling technique up to then generally disregarded as being too cumbersome for the technical limitations of game engines. Shortly after, EA DICE announced that its 2015 flagship title Star Wars Battlefront would be adopting photogrammetry extensively not only to capture key props and costumes but also to recreate key locations beloved to Star Wars fans. Since then, the games industry has been flooded with articles, tutorials, and new software dedicated to the use of photogrammetry to create 3-D game assets. This article examines the key developments in the field since 2014, how the technique is being adopted by game studios, what are the current and future trends, and how the use of photogrammetry is likely to disrupt the well-established game development pipelines.

show abstract

Just‐in‐time, viable, 3‐D avatars from scans

Cited by 20 publications

References 31 publications

Affordable But Not Cheap: A Case Study of the Effects of Two 3D-Reconstruction Methods of Virtual Humans

Affordable But Not Cheap: A Case Study of the Effects of Two 3D-Reconstruction Methods of Virtual Humans

A Robust Multi‐View System for High‐Fidelity Human Body Shape Reconstruction

Use of Photogrammetry in Video Games: A Historical Overview

Contact Info

Product

Resources

About