Toward a compelling sensation of telepresence: demonstrating a portal to a distant (static) office

Abstract

In 1998 we introduced the idea for a project we call the Office of the Future. Our long-term vision is to provide a better every-day working environment, with high-fidelity scene reconstruction for life-sized 3D tele-collaboration. In particular, we want a true sense of presence with our remote collaborator and their real surroundings. The challenges related to this vision are enormous and involve many technical tradeoffs. This is true in particular for scene reconstruction. Researchers have been striving to achieve real-time approaches, and while they have made respectable progress, the limitations of conventional technologies relegate them to relatively low resolution in a restricted volume.In this paper we present a significant step toward our ultimate goal, via a slightly different path. In lieu of low-fidelity dynamic scene modeling we present an exceedingly high fidelity reconstruction of a real but static office. By assembling the best of available hardware and software technologies in static scene acquisition, modeling algorithms, rendering, tracking and stereo projective display, we are able to demonstrate a portal to a real office, occupied today by a mannequin, and in the future by a real remote collaborator. We now have both a compelling sense of just how good it could be, and a framework into which we will later incorporate dynamic scene modeling, as we continue to head toward our ultimate goal of 3D collaborative, telepresence.

How Do Background and Remote User Representations Affect Social Telepresence in Remote Collaboration?: A Study with Portal Display, a Head Pose-Responsive Video Teleconferencing System

“…While this setup is recommended to improve the user experience, it remains optional. Notably, analogous techniques leveraging desk properties to craft shared space illusions have been documented in other teleconferencing systems [4,12,14,27].…”

RemoteTouch: Enhancing Immersive 3D Video Communication with Hand Touch

“…Early methods [4,21,59] enable 3D video communication by capturing the geometry and appearance of the participants and rendering them into one shared virtual environments. Later, a number of approaches have been introduced to enhance the gaze contacts between the participants in various meeting setups [22,32,34,37,38,51,56,73,77,81] or enhance the rendering quality of human characters [16,17,23,47,54,85]. Most recently, Lawrence et al presented Project Starline [39] for high fidelity 3D communication between two participants at different locations with the help of dedicated hardware setup for 3D video capturing and light field display.…”

An Experiment on Multi-Video Transmission with Multipoint Tiled Display Wall

“…Ideally, there exists a seamless continuum between the user's experience of local and remote space within the application. The communication system using the tele-immersion environment includes Tele-Cubicles [8] and 3-D video-conference system by 3-D modeling and display techniques [9]- [11]. Recently, IPT (Immersive Projection Technology) such as the CAVE system [12] has become popular, and teleimmersive virtual environments are constructed by using IPT.…”