DeepFit: 3D Surface Fitting via Neural Network Weighted Least Squares

Ben-Shabat, Yizhak; Gould, Stephen J.

doi:10.1007/978-3-030-58452-8_2

Cited by 48 publications

(26 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Next, two global features v X and v Y are obtained by passing G X and G Y to the point-wise maxpooling layers. Despite there are many more advanced feature extractor candidates [30], [31], [32], experimental results show the simple encoder can work well in our study, and to compare the performances of different encoders is not the target of this study.…”

Section: B Shape-interrelated Featuresmentioning

confidence: 92%

AlignBodyNet: Deep Learning-Based Alignment of Non-Overlapping Partial Body Point Clouds From a Single Depth Camera

Munteanu

2023

IEEE Trans. Instrum. Meas.

View full text Add to dashboard Cite

This paper proposes a novel deep learning framework to generate omnidirectional 3D point clouds of human bodies by registering the front-and back-facing partial scans captured by a single depth camera. Our approach does not require calibration-assisting devices, canonical postures, nor does it make assumptions concerning an initial alignment or correspondences between the partial scans. This is achieved by factoring this challenging problem into (i) building virtual correspondences for partial scans, and (ii) implicitly predicting the rigid transformation between the two partial scans via the predicted virtual correspondences. In this study, we regress the SMPL vertices from the two partial scans for building the virtual correspondences. The main challenges are (i) estimating the body shape and pose under clothing from single partial dressed body point clouds, and (ii) the predicted bodies from front-and back-facing inputs required to be the same. We, thus, propose a novel deep neural network dubbed AlignBodyNet that introduces shape-interrelated features and a shape-constraint loss for resolving this problem. We also provide a simple yet efficient method for generating real-world partial scans from complete models, which fills the gap in the lack of quantitative comparisons based on the real-world data for various studies including partial registration, shape completion, and view synthesis. Experiments based on synthetic and real-world data show that our method achieves state-of-the-art performance in both objective and subjective terms.

show abstract

Section: B Shape-interrelated Featuresmentioning

confidence: 92%

AlignBodyNet: Deep Learning-Based Alignment of Non-Overlapping Partial Body Point Clouds From a Single Depth Camera

Munteanu

2023

IEEE Trans. Instrum. Meas.

View full text Add to dashboard Cite

show abstract

“…We adopted the improved model of DeepFit [20] to achieve nonlinear mapping. DeepFit uses PointNet to extract features and a multi-layer perceptron model to predict the weight of points in the fitting.…”

Section: Cnn-based Parameter Calculationmentioning

confidence: 99%

Three Dimensional Metal-Surface Processing Parameter Generation Through Machine Learning-Based Nonlinear Mapping

Zhu

Dong

Shen

et al. 2023

Tsinghua Sci. Technol.

View full text Add to dashboard Cite

The accuracy and efficiency of three-dimensional (3D) surface forming, which directly affects the cycle and quality of production, is important in manufacturing. In practice, given the uncertainty of metal plate springback, an error exists between the actual plate and the target surface, which creates a nonlinear mapping from computer aided design models to bending surfaces. Technicians need to reconfigure parameters and process a surface multiple times to delicately control springback, which greatly wastes human and material resources. This study aims to address the springback control problem to improve the efficiency and accuracy of sheet metal forming.A basic computation approach is proposed based on the DeepFit model to calculate the springback value in 3D surface bending. To address the sample data shortage problem, we put forward an advanced approach by combining a deep learning model with case-based reasoning (CBR). Next, a multi-model fused bending parameter generation framework is devised to implement the advanced springback computation approach through surface data preprocessing, CBR-based model matching, convolution neural network-based machining surface generation, and bending parameter generation with a series of model transformations. Moreover, the proposed approaches and the framework are verified by considering saddle surface processing as an example. Overall, this study provides a new idea to improve the accuracy and efficiency of surface processing.

show abstract

“…Other approaches learn objectlevel [33,38,47] or scene-level priors [12,13,26,49]. Most reconstruction approaches directly reconstruct a meshed surface geometry, though some works [3,4,20,31] first predict point set normals to subsequently reconstruct the geometry via PSR [28,29]. However, such methods fail to handle large levels of noise, since they are unable to move points or selectively ignore outliers.…”

Section: Learning-based 3d Reconstruction From Point Cloudsmentioning

confidence: 99%

“…The learning rate is set to 2 × 10 −3 at the initial resolution of 32 3 with a decay of 0.7 after every increase of the grid resolution. Moreover, we run 1000 iterations at every grid resolution of 32 3 , 64 3 and 128 3 , and 200 iterations for 256 3 . 20000 source points and normals are used by our method to represent the final shapes for all objects.…”

Section: B Implementation Detailsmentioning

confidence: 99%

“…This differential bridge between points, indicator functions, and meshes allows us to represent shapes as oriented point clouds. We therefore call this Representations Points [16] Voxels [11] Meshes [57] Patches [19] Implicits [38] SAP (Ours) GT Efficiency Grid Eval Time (128 3 ) n/a n/a n/a n/a 0.33s 0.012s Priors Easy Initialization Quality Watertight No Self-intersection n/a n/a Topology-Agnostic Table 1: Overview of Different Shape Representations. Shape-As-Points produces higher quality geometry compared to other explicit representations [11,16,19,57] and requires significantly less inference time for extracting geometry compared to neural implicit representations [38].…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Shape As Points: A Differentiable Poisson Solver

Peng¹,

Jiang²,

Liao³

et al. 2021

Preprint

View full text Add to dashboard Cite

In recent years, neural implicit representations gained popularity in 3D reconstruction due to their expressiveness and flexibility. However, the implicit nature of neural implicit representations results in slow inference time and requires careful initialization. In this paper, we revisit the classic yet ubiquitous point cloud representation and introduce a differentiable point-to-mesh layer using a differentiable formulation of Poisson Surface Reconstruction (PSR) that allows for a GPUaccelerated fast solution of the indicator function given an oriented point cloud. The differentiable PSR layer allows us to efficiently and differentiably bridge the explicit 3D point representation with the 3D mesh via the implicit indicator field, enabling end-to-end optimization of surface reconstruction metrics such as Chamfer distance. This duality between points and meshes hence allows us to represent shapes as oriented point clouds, which are explicit, lightweight and expressive. Compared to neural implicit representations, our Shape-As-Points (SAP) model is more interpretable, lightweight, and accelerates inference time by one order of magnitude. Compared to other explicit representations such as points, patches, and meshes, SAP produces topology-agnostic, watertight manifold surfaces. We demonstrate the effectiveness of SAP on the task of surface reconstruction from unoriented point clouds and learning-based reconstruction. * Work done while at UC Berkeley.Preprint. Under review.

show abstract

DeepFit: 3D Surface Fitting via Neural Network Weighted Least Squares

Cited by 48 publications

References 17 publications

AlignBodyNet: Deep Learning-Based Alignment of Non-Overlapping Partial Body Point Clouds From a Single Depth Camera

AlignBodyNet: Deep Learning-Based Alignment of Non-Overlapping Partial Body Point Clouds From a Single Depth Camera

Three Dimensional Metal-Surface Processing Parameter Generation Through Machine Learning-Based Nonlinear Mapping

Shape As Points: A Differentiable Poisson Solver

Contact Info

Product

Resources

About