HRBF-Fusion: Accurate 3D Reconstruction from RGB-D Data Using On-the-fly Implicits

Abstract

Reconstruction of high-fidelity 3D objects or scenes is a fundamental research problem. Recent advances in RGB-D fusion have demonstrated the potential of producing 3D models from consumer-level RGB-D cameras. However, due to the discrete nature and limited resolution of their surface representations (e.g., point- or voxel-based), existing approaches suffer from the accumulation of errors in camera tracking and distortion in the reconstruction, which leads to an unsatisfactory 3D reconstruction. In this paper, we present a method using on-the-fly implicits of Hermite Radial Basis Functions (HRBFs) as a continuous surface representation for camera tracking in an existing RGB-D fusion framework. Furthermore, curvature estimation and confidence evaluation are coherently derived from the inherent surface properties of the on-the-fly HRBF implicits, which devote to a data fusion with better quality. We argue that our continuous but on-the-fly surface representation can effectively mitigate the impact of noise with its robustness and constrain the reconstruction with inherent surface smoothness when being compared with discrete representations. Experimental results on various real-world and synthetic datasets demonstrate that our HRBF-fusion outperforms the state-of-the-art approaches in terms of tracking robustness and reconstruction accuracy.

LISR: Learning Linear 3D Implicit Surface Representation Using Compactly Supported Radial Basis Functions

“…Parametric Implicit Surfaces : Another categories of algorithms parameterize the implicit surface representation and find optimal parameters either using classical optimization approaches (Huang, Carr, and Ju 2019;Blane et al 2000;Rouhani and Sappa 2010;Macêdo, Gois, and Velho 2011;Zhao et al 2021) or using learning-based approaches (Yavartanoo et al 2021;Xu et al 2022). Yavartanoo et al represents the implicit surface by a finite low-degree algebraic polynomial and learns the co-efficient of the algebraic polynomial (Yavartanoo et al 2021).…”

“…This approach suffers from the fact that the low-degree polynomials fail to model the high curvature region of the surface. The radial basis functions have been used classically to model complex implicit surfaces (Macêdo, Gois, and Velho 2011;Liu et al 2016;Xu et al 2022) however have not been utilized for surface reconstruction in a learning-based framework. In this work, we propose an RBF-based representation to model implicit representation and learn the parameters of this linear representation using a neural network.…”

“…There exist works in the literature that aim to parameterize implicit surfaces (Park et al 2019;Macêdo, Gois, and Velho 2011;Yavartanoo et al 2021). One category of works aims at modeling the implicit representation by a function parameterized by the neural networks known as neural implicit (Park et al 2019;Saito et al 2019;Michalkiewicz et al 2019b;Xu et al 2022). Yavartanoo et al represents the implicit surface by a finite low-degree algebraic polynomial and learns the co-efficient of the algebraic polynomial (Yavartanoo et al 2021).…”

“…This suffers from the fact that the low-degree polynomials fail to model the high curvature region of the surface. The radial basis functions have been used classically to model complex implicit surfaces (Macêdo, Gois, and Velho 2011;Liu et al 2016;Xu et al 2022).…”