PSDF Fusion: Probabilistic Signed Distance Function for On-the-fly 3D Data Fusion and Scene Reconstruction

Dong, Wei; Wang, Qiuyuan; Wang, Xin; Zha, Hongbin

doi:10.1007/978-3-030-01240-3_43

Cited by 42 publications

(33 citation statements)

References 31 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Existing state-of-the-art methods can be classified into two categories: online reconstructions [34]- [38], i.e., dense simultaneous localization and mapping (SLAM); and offline reconstructions [39]- [43], which have higher accuracy. To obtain high-quality 3D reconstruction, Bundlefusion [38] uses additional color features for registration and global bundle adjustment for obtaining precise scene geometry; Choi et al [39] and Zeng et al [40] reconstructed local smooth scene fragments and globally aligned them together with 3D features.…”

Section: Related Work a Rgb-d Scene 3d Reconstructionmentioning

confidence: 99%

High-Quality 3D Reconstruction With Depth Super-Resolution and Completion

Gao

2019

IEEE Access

View full text Add to dashboard Cite

The 3D reconstruction is an important topic in computer vision with many applications, such as robotics and augmented reality. Since the raw depth images captured by consumer RGB-D cameras are often low resolution (LR), noisy, and incomplete. How to obtain high-quality 3D models with a consumer RGB-D camera is still a challenge for the existing systems. In this paper, we propose a new depth super-resolution and completion method implemented in a deep learning framework and build a high-quality 3D reconstruction system. We first improve the resolution of LR depth image with a depth super-resolution network and remove the outliers in high-resolution (HR) depth image based on gradient saliency. To further enhance the quality of HR depth image with the guide of HR color image, we learn surface normal and occlusion boundary images from the corresponding HR color image through two deep fully convolutional networks. In particular, the blurriness of HR color image is also detected and pixel-wise quantized. Finally, we obtain a completed HR depth image by optimizing the HR depth image with the surface normal, occlusion boundary, and color image blurriness. We have carried out qualitative and quantitative evaluations with baseline methods on public datasets. The experimental results demonstrate that our method has better performance both on single depth image enhancement and 3D reconstruction.

show abstract

Section: Related Work a Rgb-d Scene 3d Reconstructionmentioning

confidence: 99%

High-Quality 3D Reconstruction With Depth Super-Resolution and Completion

Gao

2019

IEEE Access

View full text Add to dashboard Cite

show abstract

“…A similar model with longrange ray-based visibility constraints was used in [47,46], although these methods are not real-time capable. Recently, PSDF Fusion [15] demonstrated a combination of probabilistic modeling and a TSDF scene representation. However, they also assume a Gaussian error distribution of the input depth values.…”

Section: Related Workmentioning

confidence: 99%

Learned Semantic Multi-Sensor Depth Map Fusion

Rozumnyi¹,

Cherabier²,

Pollefeys³

et al. 2019

2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW)

View full text Add to dashboard Cite

Volumetric depth map fusion based on truncated signed distance functions has become a standard method and is used in many 3D reconstruction pipelines. In this paper, we are generalizing this classic method in multiple ways: 1) Semantics: Semantic information enriches the scene representation and is incorporated into the fusion process. 2) Multi-Sensor: Depth information can originate from different sensors or algorithms with very different noise and outlier statistics which are considered during data fusion. 3) Scene denoising and completion: Sensors can fail to recover depth for certain materials and light conditions, or data is missing due to occlusions. Our method denoises the geometry, closes holes and computes a watertight surface for every semantic class. 4) Learning: We propose a neural network reconstruction method that unifies all these properties within a single powerful framework. Our method learns sensor or algorithm properties jointly with semantic depth fusion and scene completion and can also be used as an expert system, e.g. to unify the strengths of various photometric stereo algorithms. Our approach is the first to unify all these properties. Experimental evaluations on both synthetic and real data sets demonstrate clear improvements.

show abstract

“…With the advent of Kinectstyle active depth sensors, the KinectFusion [1] algorithm permits dense volumetric reconstruction of the scene in realtime, enabling mesh models output for physics-based augmented reality (AR) [24] and 3D printing [25]. Improved frameworks have then been proposed in the aspects of memory efficiency [26]- [28], large space representation [8], [11], [27], [29], camera trajectory accuracy with loop closure detection and optimization [12], [27], [30], and scene representation such as surfels [31] or hybrid data structure [32].…”

Section: Related Workmentioning

confidence: 99%

CuFusion2: Accurate and Denoised Volumetric 3D Object Reconstruction Using Depth Cameras

Zhang

2018

Preprint

View full text Add to dashboard Cite

3D object reconstruction from depth image streams using Kinect-style depth cameras have been extensively studied. We propose an approach for accurate camera tracking and volumetric dense surface reconstruction, assuming a known cuboid reference object is present in the scene. Our contribution is threefold: (a) we keep drift-free camera pose tracking by incorporating the 3D geometric constraints of the cuboid reference object into the image registration process; (b) on the problem of depth stream fusion, we reformulate it as a binary classification problem, enabling high fidelity of surface reconstruction, especially in concave zones of the objects; (c) we further present a surface denoising strategy, facilitating the generation of  noise-free triangle mesh, making the models more suitable for 3D printing and other applications. We extend our public dataset CU3D with several fresh image sequences, test our algorithm on these sequences and compare them with other state-of-the-art algorithms. Both our dataset and algorithm are available as open-source at https://github.com/zhangxaochen/CuFusion, for other researchers to reproduce and verify our results.

show abstract

PSDF Fusion: Probabilistic Signed Distance Function for On-the-fly 3D Data Fusion and Scene Reconstruction

Cited by 42 publications

References 31 publications

High-Quality 3D Reconstruction With Depth Super-Resolution and Completion

High-Quality 3D Reconstruction With Depth Super-Resolution and Completion

Learned Semantic Multi-Sensor Depth Map Fusion

CuFusion2: Accurate and Denoised Volumetric 3D Object Reconstruction Using Depth Cameras

Contact Info

Product

Resources

About