DeepSpline: Data-Driven Reconstruction of Parametric Curves and Surfaces

Gao, Jun; Tang, Chengcheng; Ganapathi-Subramanian, Vignesh; Huang, Jiahui; Su, Hao; Guibas, Leonidas J.

doi:10.48550/arxiv.1901.03781

Cited by 18 publications

(19 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Typically, they predefined the topology of the shape, e.g. equivalent to a sphere [5,25,54], or a union of primitives [19,43,53] or a set of segmented parts [50,61,62]. As a result, they can not model a distribution of shapes with complex topology variations.…”

Section: Voxel-based Methodsmentioning

confidence: 99%

Deep Marching Tetrahedra: a Hybrid Representation for High-Resolution 3D Shape Synthesis

Shen¹,

Gao²,

Yin³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

We introduce DMTET, a deep 3D conditional generative model that can synthesize high-resolution 3D shapes using simple user guides such as coarse voxels. It marries the merits of implicit and explicit 3D representations by leveraging a novel hybrid 3D representation. Compared to the current implicit approaches, which are trained to regress the signed distance values, DMTET directly optimizes for the reconstructed surface, which enables us to synthesize finer geometric details with fewer artifacts. Unlike deep 3D generative models that directly generate explicit representations such as meshes, our model can synthesize shapes with arbitrary topology. The core of DMTET includes a deformable tetrahedral grid that encodes a discretized signed distance function and a differentiable marching tetrahedra layer that converts the implicit signed distance representation to the explicit surface mesh representation. This combination allows joint optimization of the surface geometry and topology as well as generation of the hierarchy of subdivisions using reconstruction and adversarial losses defined explicitly on the surface mesh. Our approach significantly outperforms existing work on conditional shape synthesis from coarse voxel inputs, trained on a dataset of complex 3D animal shapes. Project page: https://nv-tlabs.github.io/DMTet/.Recently, neural implicit representations [8,39,42,51], which use a neural network to implicitly represent a shape via a signed distance field (SDF) or an occupancy field (OF), have emerged as an effective 3D representation. Neural implicits have the benefit of representing complex geometry 35th Conference on Neural Information Processing Systems (NeurIPS 2021).

show abstract

Section: Voxel-based Methodsmentioning

confidence: 99%

Deep Marching Tetrahedra: a Hybrid Representation for High-Resolution 3D Shape Synthesis

Shen¹,

Gao²,

Yin³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…The two most common splines, i.e. the cubic Bezier spline and the uniform B-Spline [27,12], are defined by control points which do not lie on the curve, which could potentially confuse an annotator that needs to make edits. Following [32], we use the centripetal Catmull-Rom spline (CRS) [35], which has control points along the curve.…”

Section: Spline Parametrizationmentioning

confidence: 99%

Fast Interactive Object Annotation With Curve-GCN

Ling

Gao

Kar

et al. 2019

2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

Self Cite

272

193

View full text Add to dashboard Cite

Manually labeling objects by tracing their boundaries is a laborious process. In [7,2], the authors proposed Polygon-RNN that produces polygonal annotations in a recurrent manner using a CNN-RNN architecture, allowing interactive correction via humans-in-the-loop. We propose a new framework that alleviates the sequential nature of Polygon-RNN, by predicting all vertices simultaneously using a Graph Convolutional Network (GCN). Our model is trained endto-end. It supports object annotation by either polygons or splines, facilitating labeling efficiency for both line-based and curved objects. We show that Curve-GCN outperforms all existing approaches in automatic mode, including the powerful PSP-DeepLab [8,23] and is significantly more efficient in interactive mode than Polygon-RNN++. Our model runs at 29.3ms in automatic, and 2.6ms in interactive mode, making it 10x and 100x faster than Polygon-RNN++.

show abstract

“…Egiazarian et al [6] propose a transformer-based architecture for translating technical line drawings to vector parameters. Gao et al [8] produce parametric curves utilizing the extracted image features and a designed hierarchical recurrent network. Guo et al [10] sub-divide the lines into partial curves employing a deep network, and reconstruct the topology at junctions.…”

Section: Related Workmentioning

confidence: 99%

“…The first category of works is based on pre-designed algorithms, which analyzes pixels and calculates parameters to construct vector graphics (e.g., [2,5,14,16,21,22,31]). The other category of works is based on deep learning (DL) (e.g., [4,6,8,18,23,[27][28][29]), which trains neural models to produce vector graphics utilizing the features of target raster images. However, the DL-based methods typically vectorize an entire image in one step, and the one-step manner makes the DL model cannot handle too many parameters of vector format accurately.…”

Section: Introductionmentioning

confidence: 99%

Vectorization of Raster Manga by Deep Reinforcement Learning

Sheng¹,

Niu²,

Li³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

Manga is a popular Japanese-style comic form that consists of black-and-white stroke lines. Compared with images of real-world scenarios, the simpler textures and fewer color gradients of mangas are the extra natures that can be vectorized. In this paper, we propose Mang2Vec, the first approach for vectorizing raster mangas using Deep Reinforcement Learning (DRL). Unlike existing learning-based works of image vectorization, we present a new view that considers an entire manga as a collection of basic primitives "stroke line", and the sequence of strokes lines can be deep decomposed for further vectorization. We train a designed DRL agent to produce the most suitable sequence of stroke lines, which is constrained to follow the visual feature of the target manga. Next, the control parameters of strokes are collected to translated to vector format. To improve our performances on visual quality and storage size, we further propose an SA reward to generate accurate stokes, and a pruning mechanism to avoid producing error and redundant strokes. Quantitative and qualitative experiments demonstrate that our Mang2Vec can produce impressive results and reaches the state-of-the-art level.

show abstract

DeepSpline: Data-Driven Reconstruction of Parametric Curves and Surfaces

Cited by 18 publications

References 18 publications

Deep Marching Tetrahedra: a Hybrid Representation for High-Resolution 3D Shape Synthesis

Deep Marching Tetrahedra: a Hybrid Representation for High-Resolution 3D Shape Synthesis

Fast Interactive Object Annotation With Curve-GCN

Vectorization of Raster Manga by Deep Reinforcement Learning

Contact Info

Product

Resources

About