Live 3D scenography system and tangible 3D vision

Abstract

The present media seen on the network are no more than pictures and movies. No new media is output. Conversely, the expectation for 3D images has risen in various fields such as entertainment and medicine. However, it is necessary to learn how to use special devices and software in order to make 3D images on the computer, and this prevented users from using 3D image content. Therefore, we have developed the "Live 3D Scenography System and Tangible 3D Vision." ¾ ÜÔÓ× Ø ÓÒOur system has two main parts, "Imaging Section" and "Haptic Section".The imaging section is the system for photographing and distributing 3D videos in real-time. With this system, we were able to take many types of 3D videos, shot at a short distance and using table objects, we found a strong desire to touch the 3D images.Therefore we added the haptic section. The haptic section is a process for providing haptic sense. With these two parts together, the user can experience a new sense that is able to touch stereoscopic real-time videos. Figure 1: The Processing Flow Figure 1 shows the processing flow of the system.Our new system is composed of a stereo camera, a processing computer, a haptic device controller, a mechanical glove-and-arm type of haptic device and a parallax-barrier type of 3D monitor. £During the imaging section, stereoscopic images taken by the stereo camera are synthesized as a 3D image and sent to a 3D monitor. The entire process was done in real-time to provide live videos. The size of the image is XGA and the movie's frame rate is 30 fps. To make the system easy to use and low-cost, all processing to create the 3D synthesis is implemented using software only, without special hardware. Moreover, we added special software processes to improve the 3D effect, and it brings realistic stereoscopic live videos that have not been created before.The haptic section is the additional stage to provide haptic sense. At first, the processing computer extracts the shape of the target from the stereoscopic images. Next, the haptic device controller creates haptic sense from the shape data mate at processing computer. Then the haptic device presents haptic sense and the 3D monitor displays stereoscopic real-time video.Those two types of 3D virtual spaces, stereoscopic space and haptic 3D space, are synchronized exactly, and the user can touch the stereoscopic real-time video displayed in front of a 3D monitor, shown in Figure 2. The video is not 3D-CG but live-action shot.Figure 2: This user is touching a approaching ball that is displayed in front of the 3D monitor. ¿ ÓÒÐÙ× ÓÒWe have developed the "Live 3D Scenography System and Tangible 3D Vision." The system creates a new media on the network.Applying this system to network communication tools, we think a new type of visual haptic communication will be born.In the future, we will develop a 3D touchable video-phone and some applications based on this system, and will expand the possibility of the new media.

Tangible-3d

“…Our first step was developing the tactile 3D video system named the Tangible-3D: Rolling Ball Model [3]. In this system, the user can feel the shape and weight of a rolling ball that is displayed in front of the 3D monitor as 3D video.…”