Augmented Virtual Teleportation for High-Fidelity Telecollaboration

Rhee, Taehyun; Thompson, Stephen A.; Medeiros, Daniel; Anjos, Rafael dos; Chalmers, Andrew

doi:10.1109/tvcg.2020.2973065

Cited by 62 publications

(35 citation statements)

References 71 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…360-videos shown in HMDs allow for a high sense of telepresence [39,62], and MR in 360videos enables user interactions in the captured environment [48,52,62]. Recently, augmented virtual teleportation (AVT) [53] presented a platform for remote collaboration in MR using a 360-camera as a teleportal. A remote VR traveller using an HMD collaborates with an AR host in a MR collaboration space while eliciting a high sense of spatial presence and co-presence in the captured environment as if they were in the same collaboration space.…”

Section: Motivationmentioning

confidence: 99%

“…This becomes problematic when multiple users simultaneously experience the virtual environment, which limits non-verbal communication between users and compromising the overall immersive experience. Furthermore, most approaches found in the literature are dependent on head-mounted displays (HMDs) [21,52,53] that provide a complete immersion experience that shuts out the physical world. As HMD obscures each user's face, it provides a high level of presence.…”

Section: Motivationmentioning

confidence: 99%

“…One example of this is Hoppin' World [2], which is a commercial VR application that allows multiple users wearing HMDs to explore 360-videos together while communicating each other's avatars in the recorded video only. Another recent example is the Augmented Virtual Teleportation (AVT) [53] that uses a 360-camera as a portal to teleport an immersed user into a remote location in real time, where the one can collaborate with a local collaborator using augmented virtual objects that match the environment light conditions. This setup provides a high sense of presence and co-presence to both remote and local users, though at the cost of limited mobility for the remote user due to 360-cameras limitations.…”

Section: Remote Multi-user Virtual Environmentmentioning

confidence: 99%

See 2 more Smart Citations

TeleGate: Multi-user Semi-teleportation for Remote Collaboration in Mixed Reality 360-Videos

Kim¹

2021

Preprint

View full text Add to dashboard Cite

Telepresence systems enable people to feel present in a remote space while their bodies remain in their local space. To enhance telepresence, the remote environment needs to be captured and visualised in an immersive way. For instance, 360-degree videos (360-videos) shown on head-mounted displays (HMDs) provide high fidelity telepresence in a remote place. Mixed reality (MR) in 360-videos enables interactions with virtual objects blended in the captured remote environment while it allows telepresence only for a single user wearing HMD. For this reason, it has limitations when multiple users want to experience telepresence together and naturally collaborate within a teleported space. This thesis presents TeleGate, a novel multi-user teleportation platform for remote collaboration in a MR space. TeleGate provides "semi-teleportation" into the MR space using large-scale displays, acting as a bridge between the local physical communication space and the remote collaboration space created by MR with captured 360-videos. Our proposed platform enables multi-user semi-teleportation to perform collaborative tasks in the remote MR collaboration (MRC) space while allowing for natural communication between collaborators in the same local physical space. We implemented a working prototype of TeleGate and then conducted a user study to evaluate our concept of semi-teleportation. We measured the spatial presence, social presence while participants performed remote collaborative tasks in the MRC space. Additionally, we also explored the different control mechanisms within the platform in the remote MR collaboration scenario. In conclusion, TeleGate enabled multiple co-located users to semi-teleport together using large-scale displays for remote collaboration in MR 360-videos.

show abstract

Section: Motivationmentioning

confidence: 99%

Section: Motivationmentioning

confidence: 99%

Section: Remote Multi-user Virtual Environmentmentioning

confidence: 99%

See 1 more Smart Citation

TeleGate: Multi-user Semi-teleportation for Remote Collaboration in Mixed Reality 360-Videos

Kim¹

2021

Preprint

View full text Add to dashboard Cite

show abstract

“…It requires higher computational complexity and economic costs to render high-resolution content on these devices. Furthermore, different categories of immersive display devices demand a content creation system that can work across multiple categories of display devices [6], [7]. Therefore, the design of a consumer-grade and economically viable real-time 360° stitching system involves an increase in system performance along with reduced cost and complexities.…”

Section: B Immersive Content Consumptionmentioning

confidence: 99%

“…Therefore, for a quadrilateral with top left corner at ,1 , the top right, bottom left and bottom right corners will be ,2 , ,1+ and ,2+ . We use inverse of calibration parameters for each camera to transform these corner points to each camera frame, as in (6).…”

Section: ) Lookup Tables and Blend Masksmentioning

confidence: 99%

A High-Quality VR Calibration and Real-Time Stitching Framework Using Preprocessed Features

et al. 2020

View full text Add to dashboard Cite

Virtual Reality (VR) contents comprise 360°×180° seamless panoramic videos, stitched from multiple overlapping video streams. A recent increase in demand of VR content has resulted in the supply of commercially available solutions which, though cheap, lack scalability and quality. In this paper, we propose an end-to-end VR system for stitching full spherical contents. The VR system is composed of camera rig calibration and stitching modules. The calibration module performs geometric alignment of camera rig. The stitching module transforms texture from camera or video stream into VR stream using lookup tables (LUTs) and blend masks (BMs). In this work, our main contribution is improvement of stitching quality. First, we propose a feature preprocessing method that filters out inconsistent, error-prone features. Secondly, we propose a geometric alignment method that outperforms state-of-the-art VR stitching solutions. We tested our system on diverse image sets and obtained state-of-the-art geometric alignment. Moreover, we achieved real-time stitching of camera and video streams up to 120 fps at 4K resolution. After stitching, we encode VR content for IP multicasting.

show abstract

EdTech.update – New Tools fürs New Learning

Foelsing¹,

Schmitz

2021

New Work Braucht New Learning

View full text Add to dashboard Cite

Augmented Virtual Teleportation for High-Fidelity Telecollaboration

Abstract: Fig. 1. Augmented Virtual Teleportation -an asymmetric platform for remote collaboration. From the left: a remote traveler wearing a VR HMD, the Mixed Reality collaboration space seen in their display, the space shown on an AR display, and the local host.

Cited by 62 publications

References 71 publications

TeleGate: Multi-user Semi-teleportation for Remote Collaboration in Mixed Reality 360-Videos

TeleGate: Multi-user Semi-teleportation for Remote Collaboration in Mixed Reality 360-Videos

A High-Quality VR Calibration and Real-Time Stitching Framework Using Preprocessed Features

EdTech.update – New Tools fürs New Learning

Contact Info

Product

Resources

About