Replacing Projective Data Association with Lucas-Kanade for KinectFusion

Abstract

Abstract-We propose to overcome a significant limitation of the KinectFusion algorithm, namely, its sole reliance upon geometric information to estimate camera pose. Our approach uses both geometric and color information in a direct manner that uses all the data in order to perform the association of data between two RGBD point clouds. Data association is performed by aligning the two color images associated with the two point clouds by estimating a projective warp using the Lucas-Kanade algorithm. This projective warp is then used to create a correspondence map between the two point clouds, which is then used as the data association for a point-to-plane error minimization. This approach to correspondence allows camera tracking to be maintained through areas of low geometric features. We show that our proposed LKDA data association technique enables accurate scene reconstruction in environments in which low geometric texture causes the existing approach to fail, while at the same time demonstrating that the new technique does not adversely affect results in environments in which the existing technique succeeds.

Focus plus context visualization based on volume clipping for markerless on-patient medical data visualization

“…Depth-based tracking algorithms (e.g., ICP) are dependent on the presence of geometric information on the scene [70]. Some ROI, such as the arm and leg, do not have much variation in the depth values captured by the sensor between different viewpoints.…”

Feature-based RGB-D camera pose optimization for real-time 3D reconstruction

“…Top: four real input point clouds scanned using different views of a white wall with a painting. Bottom: results of stitching using camera poses provided by the Lucas-Kanade method [6], voxel-hashing [2], ElasticFusion [9], and our method. and offline reconstruction techniques, and briefly introduce camera pose optimization in some other relevant areas.…”

“…While KinectFusion runs a frameto-model ICP process with vertex correspondence obtained by projective data association, Peasley and Birchfield [6] improved it by providing ICP with a better initial guess and correspondence based on a warp transformation between consecutive RGB images.…”

“…To solve the aforementioned limitation, a common strategy adopted by most subsequent online reconstruction methods is to introduce photometric data into the ICP-based framework to estimate camera poses by maximizing the consistency of geometric information as well as color information between two adjacent frames [2,[5][6][7][8][9][10][11]. However, even though an ICP-based framework can effectively deal with RGB-D data with small shifts, it solves a non-linear minimization problem and always converges to a local minimum near the initial input because of the small angle assumption [4].…”

Procedurally generated virtual reality from 3D reconstructed physical space

“…The appearance of low-cost range sensors, such as the Microsoft Kinect, has provided easy access to fast and robust 3D reconstruction. KinectFusion [Newcombe et al 2011] and its variants [Peasley and Birchfield 2013;Qayyum et al 2013] are among the most popular techniques for hand-held scanning of indoor scenes. More recently, Googles Project Tango 3 tablet with an integrated depth camera and inertial sensors has been used for virtual and augmented reality applications [Nassani et al 2015].…”