MoveBox: Democratizing MoCap for the Microsoft Rocketbox Avatar Library

González-Franco, Mar; Egan, Zelia; Peachey, Matthew; Antley, Angus; Randhavane, Tanmay; Panda, Payod; Zhang, Yaying; Wang, Cheng Yao; Reilly, Derek; Peck, Tabitha C.; Won, Andrea Stevenson; Steed, Anthony; Ofek, Eyal

doi:10.1109/aivr50618.2020.00026

Cited by 20 publications

(14 citation statements)

References 34 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…That being said, Gonzalez-Franco et al [23] have recently argued that rigged avatars will be key to the future of VR and its wide adoption. They also open-sourced an avatar library for research and academic purposes that can further help researchers to adopt VR for their research [24]. All the previous works emphasise the importance of avatars in virtual environments.…”

Section: Avatars In Virtual Environmentsmentioning

confidence: 99%

“…For example, many VR studies require complex setups and additional hardware (e.g., OptiTrack systems for finger tracking [34]) or Microsoft Kinects for body tracking [73]). Furthermore, there are many ways to represent users in VR (e.g., [24,29,53]), and it is often not clear which one best imitates users. It is common to embody avatars of various appearances ranging from abstract to realistic [50].…”

Section: Summary and Research Gapmentioning

confidence: 99%

“…To find answers to bystanders' interaction identification performance, we covered three different avatars (ranging from an abstract avatar to a highly realistic avatar by Microsoft Research [24]) and used a human in the real world as our baseline. We evaluated the different avatars in front of a public display that allows input with three input methods: touch, mid-air, and eye gaze.…”

Section: Preamblementioning

confidence: 99%

“…Furthermore, we use a Leap Motion Controller for the finger tracking and the integrated Tobii eyetracker (together with the Tobii XR SDK [80]) for the eye gaze movement of the avatars. For the more realistic avatars, we used an avatar provided by Microsoft Research [24] that we slightly modified based on the purpose of our research. We calculated the joint angles of the arms using Unity's Animation Rigging package and Inverse Kinematics (IK) [30] to increase realism and link the avatar's hands to the avatar's body.…”

Section: Apparatus and Implementationmentioning

confidence: 99%

“…

Figure 1: We investigate the impact of an avatar's fidelity on a bystander's performance when identifying the avatar's touch, mid-air, and eye gaze gestures. We had an abstract avatar (➋) and two more realistic avatars (➌, ➍), which are provided by Microsoft Research [24] and modified based on our investigation. We use touch, mid-air, and eye gaze gestures performed by a human in the real world (➊) as our baseline.

…”

mentioning

confidence: 99%

See 4 more Smart Citations

Observing Virtual Avatars: The Impact of Avatars’ Fidelity on Identifying Interactions

Mathis

Vaniea

Khamis

2021

Academic Mindtrek 2021

View full text Add to dashboard Cite

show abstract

Section: Avatars In Virtual Environmentsmentioning

confidence: 99%

Section: Summary and Research Gapmentioning

confidence: 99%

Section: Preamblementioning

confidence: 99%

Section: Apparatus and Implementationmentioning

confidence: 99%

“…

…”

mentioning

confidence: 99%

See 3 more Smart Citations

Observing Virtual Avatars: The Impact of Avatars’ Fidelity on Identifying Interactions

Mathis

Vaniea

Khamis

2021

Academic Mindtrek 2021

View full text Add to dashboard Cite

show abstract

Data Visualizations for Interdisciplinary Communication in HFE

Kim

Panda

2021

Advances in Industrial Design

View full text Add to dashboard Cite

Full body video-based self-avatars for mixed reality: from E2E system to user study

et al. 2023

View full text Add to dashboard Cite

In this work, we explore the creation of self-avatars through video pass-through in mixed reality (MR) applications. We present our end-to-end system, including custom MR video pass-through implementation on a commercial head-mounted display (HMD), our deep learning-based real-time egocentric body segmentation algorithm, and our optimized offloading architecture, to communicate the segmentation server with the HMD. To validate this technology, we designed an immersive VR experience where the user has to walk through a narrow tile path over an active volcano crater. The study was performed under three-body representation conditions: virtual hands, video pass-through with color-based full-body segmentation, and video pass-through with deep learning full-body segmentation. This immersive experience was carried out by 30 women and 28 men. To the best of our knowledge, this is the first user study focused on evaluating video-based self-avatars to represent the user in a MR scene. Results showed no significant differences between the different body representations in terms of presence, with moderate improvements in some Embodiment components between the virtual hands and full-body representations. Visual Quality results showed better results from the deep-learning algorithms in terms of the whole body perception and overall segmentation quality. In this study, we provide some discussion regarding the use of video-based self-avatars and some reflections on the evaluation methodology. The proposed E2E solution is in the boundary of the state-of-the-art, so there is still room for improvement before it reaches maturity. However, this solution serves as a crucial starting point for MR applications where users can feel immersed and interact with their own bodies.

show abstract

MoveBox: Democratizing MoCap for the Microsoft Rocketbox Avatar Library

Cited by 20 publications

References 34 publications

Observing Virtual Avatars: The Impact of Avatars’ Fidelity on Identifying Interactions

Observing Virtual Avatars: The Impact of Avatars’ Fidelity on Identifying Interactions

Data Visualizations for Interdisciplinary Communication in HFE

Full body video-based self-avatars for mixed reality: from E2E system to user study

Contact Info

Product

Resources

About