42.4L: Late‐News Paper: Footprint of Scalable 3D Telecommunication: Using Integral Light Field Display and Kinect‐based Capture

Abstract

A footprint of 3D telecommunication system is introduced, using integral light field display and Kinect-based capture. Two prototypes with LCD-based projection display and multi-projectors-based display, respectively, are developed to perform the 3D display. Kinect-based 3D capture is integrated in providing preliminary and real-time human model capture. This approach is scalable and demonstrated as a potential solution for market-oriented 3D telecommunication. Author KeywordsTelecommunication; 3D display; light field; Kinect; scalable IntroductionTelecommunication systems that facilitate the live information exchange among persons have become increasingly important recently. To provide better experience of communication, techniques and devices of information displays have been widely adopted to enhance the virtual reality among remote participants. Three-dimensional (3D) display is one of these devices, which has experienced an unprecedented development in both academia and industry [1,2]. Applying the latest 3D display to telecommunication systems is especially beneficial as 3D display provides natural visualization as well as important visual cues, such as eye contacts, facial motions, body gestures and etc.Parallax-based auto-stereoscopic display systems [3,4] are good candidates for 3D telecommunication systems. These low-cost and ease of deployment multi-view auto-stereoscopic displays provide high-resolution 3D images in several fixed views. However, the existing systems suffer the problems in increasing the number of viewing positions, which means in a telecommunication the participants' freedom of movement is greatly restricted. This issue has been investigated a lot, as one drawback of conventional multi-view displays that cannot provide smooth motion parallax in large viewing angular field [5].Jones et al. from University of South California developed a video conference system using a rotating 360 degree 3D display with a structured light-based 3D scanning [6]. Though this system has a 360 degree viewable scene and fast face acquisition, the rotating mechanical structure, limited display volume and complex human reconstruction algorithms restrict it for commercial telecommunications that are low-cost, compact and scalable.Far fewer literatures have discussed the market-oriented 3D telecommunications. It has several requirements. The first and the basic is the nature of communications that multiple participants are able to talk, move and act as freely as possible. It requires natural 3D visualization with smooth stereo and motion parallax, and fast human capture for virtual human representation. Another issue is the commerciality that any end-to-end 3D telecommunication systems should be low cost, compact, ease of deployment and etc. to make them be practically and widely applied in the market.To address these issues, in this paper, we present a footprint of 3D telecommunication system using scalable integral light field display with Kinect-based real-time human capture. The system is diagramed ...

21-2: Exploring 3D Interactive Performance Animation for VR/AR Applications Using Low-cost Motion Capture

“…Most current approaches rely on either markers or glove-sensing techniques, which are less comfortable than accessories-free methods. Alternatively, accurate 3D depth sensors, such as Microsoft Kinect [10][11] and Leap Motion controller [12][13], are excellent options for facilitating free-hand 3D interaction. Particularly, the infra-red (IR) laser speckle technique used by Kinect makes it a promising controller for full-body motion capture, but with limited accuracy and robustness for near-field tiny gestures, e.g., finger motions.…”

13.2: 360° Multi‐Faced Tracking and Interaction Using A Panoramic Camera

“…To tackle the problem of vertical motion parallax in the displays with horizontal parallax only in [4,5], and for face tracking and reconstruction for 3D display in [6], the authors mount a Microsoft Kinect camera directly above the screen to track users' position. However, the Kinect camera is unable to track users' faces in a much wider view field (about 70° in horizon limited), which restricts the extension use in the 360-degree display systems.…”