Virtual Agents from 360° Video for Interactive Virtual Reality

Abstract

Creating lifelike virtual humans for interactive virtual reality is a difficult task. Most current solutions rely either on crafting synthetic character models and animations, or on capturing real people with complex camera setups. As an alternative, we propose leveraging efficient learning-based models for human mesh estimation, and applying them to the popular form of immersive content that is 360°video. We demonstrate an implementation of this approach using available pre-trained models, and present user study results that show that the virtual agents generated with this method can be made more compelling by the use of idle animations and reactive verbal and gaze behavior. CCS CONCEPTS • Human-centered computing → Virtual reality; • Computing methodologies → Machine learning; Computer vision.

A semantic‐driven generation of 3D Chinese opera performance scenes

“…To reduce the sparsity problem, it is assumed that in t i only the t i−1 is determined, and the received signal c i at time i is only determined by the transmit signal t i . This assumption allows the transformation of Equation (3) into Equation (4).…”

“…The advent of digital media and 3D technology has provided new potential pathways for the inheritance and development of Chinese opera culture. 3,4 It can boost the stage presentation of opera and the artistic effect of digital opera performances. 5,6 The use of digital scenes can overcome the limitations of traditional drama in production and dissemination and create adequate conditions for the inheritance and development of Chinese opera.…”