Towards 3D Television Through Fusion of Kinect and Integral-Imaging Concepts

Hong, Seokmin; Shin, Dong-Hak; Lee, Byung-Gook; Dorado, Adrián; Saavedra, Genaro; Martı́nez-Corral, Manuel

doi:10.1109/jdt.2014.2370734

Cited by 13 publications

(5 citation statements)

References 30 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…These images are referred to as Elemental Images (EIs). 3D data can be digitally reconstructed from these multiple-perspective images using Computational Integral Imaging (CII) [8][9][10][11][12][13][14][15][16]. While Integral-Imaging can produce rich 3D data of the scene, it does not require the use of sources of illumination, as in time-of-flight cameras [17] or structured light imaging [18].…”

Section: Introductionmentioning

confidence: 99%

3D Object Detection via 2D Segmentation-Based Computational Integral Imaging Applied to a Real Video

Kadosh

Yitzhaky

2023

Sensors

View full text Add to dashboard Cite

This study aims to achieve accurate three-dimensional (3D) localization of multiple objects in a complicated scene using passive imaging. It is challenging, as it requires accurate localization of the objects in all three dimensions given recorded 2D images. An integral imaging system captures the scene from multiple angles and is able to computationally produce blur-based depth information about the objects in the scene. We propose a method to detect and segment objects in a 3D space using integral-imaging data obtained by a video camera array. Using objects’ two-dimensional regions detected via deep learning, we employ local computational integral imaging in detected objects’ depth tubes to estimate the depth positions of the objects along the viewing axis. This method analyzes object-based blurring characteristics in the 3D environment efficiently. Our camera array produces an array of multiple-view videos of the scene, called elemental videos. Thus, the proposed 3D object detection applied to the video frames allows for 3D tracking of the objects with knowledge of their depth positions along the video. Results show successful 3D object detection with depth localization in a real-life scene based on passive integral imaging. Such outcomes have not been obtained in previous studies using integral imaging; mainly, the proposed method outperforms them in its ability to detect the depth locations of objects that are in close proximity to each other, regardless of the object size. This study may contribute when robust 3D object localization is desired with passive imaging, but it requires a camera or lens array imaging apparatus.

show abstract

Section: Introductionmentioning

confidence: 99%

3D Object Detection via 2D Segmentation-Based Computational Integral Imaging Applied to a Real Video

Kadosh

Yitzhaky

2023

Sensors

View full text Add to dashboard Cite

show abstract

“…The use of 3D Integral Imaging to fuse information from multiple sources is in essence what other approaches (multi-view cameras, RGB-D) also do, albeit differently. A novel means of computing integral images has recently been proposed [14]. Although depth information for action or gesture recognition has been exploited lately [24], [6], [22], [39], [34], [23], [29], integral imaging is not used.…”

Section: Introductionmentioning

confidence: 99%

Three-Dimensional Integral Imaging for Gesture Recognition Under Occlusions

Traver

Carmona

Salvador-Balaguer

et al. 2017

IEEE Signal Process. Lett.

View full text Add to dashboard Cite

Abstract-Over the last years, three-dimensional (3D) imaging has been applied to human action and gesture recognition, usually in the form of depth maps from RGB-D sensors. An alternative which has not been explored is 3D integral imaging, aside from a recent preliminary study which shows that it can be an effective sensory modality with some advantages over the conventional monocular imaging. Since integral imaging has also been shown to be a powerful tool in other visual tasks (e.g. object reconstruction and recognition) under challenging conditions (e.g. low illumination, occlusions), and its passive longrange operation brings benefits over active close-range devices, a natural question is whether these advantages also hold for gesture recognition. Furthermore, occlusions are present in many real-world scenarios in gesture recognition, but it is an elusive problem which has scarcely been addressed. As far as we know, this work analyzes for the first time the potential of integral imaging for gesture recognition under occlusions, by comparing it to monocular imaging and to RGB-D sensory data. Empirical results corroborates the benefits of 3D integral imaging for gesture recognition, mainly under occlusions.

show abstract

“…When the integral picture is projected onto an InI display system, observers can see the 3D floating scene, which have full-parallax and quasicontinuous perspective views [1][2][3]. Many researchers and companies have applied this InI technique in many different fields [4][5][6][7][8][9][10][11][12].…”

Section: Introductionmentioning

confidence: 99%

Integral-Imaging display from stereo-Kinect capture

et al. 2017

Self Cite

View full text Add to dashboard Cite

In this paper, we propose a new approach in order to improve the quality of microimages and display them onto an integral imaging monitor. Our main proposal is based on the stereo-hybrid 3D camera system. Originally, hybrid camera system has dissimilarity itself. We interpret our method in order to equalize the hybrid sensor's characteristics and 3D data modification strategy. We generate integral image by using synthetic back-projection mapping method. Finally, we project the integral image onto our proposed display system. We illustrate this procedure with some imaging experiments in order to prove an advantage of our approach.Key words: 3D display, Integral Imaging, 3D data registration, Color transfer, Point clouds, Hybrid 3D cameras Conventional photography is fully adapted to record a three-dimensional (3D) world's scene into a two-dimensional (2D) sensing device. Although 2D images can reflect the 3D nature of scenes, they still lack of important information. Fortunately, there are techniques that are able to record 3D information from 3D scenes. Among them, integral-imaging (InI) technique has been considered as one of the most potential technologies to record and display real world scenes. The main procedure of InI is performed by placing a microlens array in front of a 2D image sensing unit. This lens array records different perspectives of the 3D scene. This is because all of the reflected light from an object is transmitted into all the lenses, which distribute the light on different pixels of the 2D sensor depending on the incidence angle of the light. Here, we name as microimage which is recorded by any microlens. The whole array of microimages is named here as the integral image. When the integral picture is projected onto an InI display system, observers can see the 3D floating scene, which have full-parallax and quasicontinuous perspective views [1][2][3]. Many researchers and companies have applied this InI technique in many different fields [4][5][6][7][8][9][10][11][12].In the meantime, many kind of depth sensing techniques have been developed to record 3D scenes [13][14][15][16]. Among them, infrared (IR) light sensing has been widely used during the last decades. Especially the Kinect device from Microsoft uses IR lighting technology for depth acquisition. Nowadays, two different versions of Kinect are released. It is well known that both sensors have many different features for obtaining a depth map. The Kinect v1 uses a structured IR light pattern emitter and IR camera to measure the depth distance from captured features at the scene [13][14]. In comparison, the Kinect v2 utilizes timeof-flight (ToF) technology, which consists of emitting IR flashes with high frequency. The IR light is reflected on most 3D surfaces and detected by a depth sensor on the Kinect v2 device. The depth distance is measured by the light's returning duration. [15][16].

show abstract

Towards 3D Television Through Fusion of Kinect and Integral-Imaging Concepts

Cited by 13 publications

References 30 publications

3D Object Detection via 2D Segmentation-Based Computational Integral Imaging Applied to a Real Video

3D Object Detection via 2D Segmentation-Based Computational Integral Imaging Applied to a Real Video

Three-Dimensional Integral Imaging for Gesture Recognition Under Occlusions

Integral-Imaging display from stereo-Kinect capture

Contact Info

Product

Resources

About