Human-Avatar Interaction in Metaverse

Abstract

The metaverse is a network of shared virtual environments where people can interact synchronously through their avatars. To enable this, it is necessary to accurately capture and recreate (physical) human motion. This is used to render avatars correctly, reflecting the motion of their corresponding users. In large-scale environments this must be done in real-time. This paper proposes a human-avatar framework with full-body motion capture. Its goal is to deliver high-accuracy capture with low computational and network overheads. It relies on a lightweight Octree data structure to record and transmit motion to other users. We conduct a user study with 22 participants and perform a preliminary evaluation of its scalability. Our user study shows that Octree with Inverse Kinematic achieves the best trade-off, achieving low delay and high accuracy. Our proposed solution delivers the lowest delay, with an average of 67ms in an environment of 8 concurrent users. It attains a 55.7% improvement over the prior techniques. CCS CONCEPTS• Human-centered computing → Mixed / augmented reality; Web-based interaction.

An Attention Economy Theme Park

“…Lam and others explain how automatic capture of human motion to recreate in virtual worlds focuses only on head and hand movements. It argues that full body tracking and avatar reconstruction requires high-resource network environments and proposes that delivery efficiency can improve in low-resource network environments by compressing data (Lam et al, 2022).…”

MetaChain: A Novel Blockchain-based Framework for Metaverse Applications

“…Note that the maximization of Q k in (11) results in reducing the round-trip VR delay D k (t) and the BCI prediction loss L φ . Our considered problem involves not only classification problem, i.e., classification of BCI signals in (8), but also decision-making problem, i.e., channel, power, and computing resource allocation problem in ( 4) and (5). The formulated problem P 1 is a Mixed-Integer Linear Programming (MILP) problem in which the power allocation variables p are continuous while the resource block allocation and classification decision variables, i.e., ρ and φ are integer variables.…”

“…Additionally, special markers were placed on the physical object used for reconstructing the virtual robotic arm to facilitate the motion capture function of the camera. In [5], the authors proposed an interaction framework that supports multiple users at a time and a higher degree of freedom for virtual human avatars. To this end, the motion capture system utilized inertial-measurement-unit (IMU) sensors attached to the human participants.…”

“…This also means less antennas, sensors can be mounted, thus limiting the communications and sensing capabilities. Furthermore, each conventional sensing technique requires specific type of sensor, e.g., camera [4], finger sensor [6], IMU sensor [5], heart rate sensor [11], and joystick [12], that may hinder the scalability and standardization of the system. For example, in some Metaverse applications such as real-time virtual fighting, fitness, and dance gaming, heart rate tracking information can help to adjust the tempo and intensity of the music, while body movement tracking information can provide more accurate and precise control over the user's dance movements.…”