GraphX-Convolution for Point Cloud Deformation in 2D-to-3D Conversion

Nguyen, Duc Thanh; Choi, Seonghwa; Kim, Woojae; Lee, Sanghoon

doi:10.1109/iccv.2019.00872

Cited by 33 publications

(19 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Recently, due to the establishment of large-scale 3D model datasets, such as ShapeNet [2] and ModelNet [62], it is popular to reconstruct complete 3D shape from a single image by utilizing shape priors modeled by deep neural networks. It also achieved various degrees of success by designing 3D shape decoders tailored for different shape representations including voxel [7,14], point cloud [10,32], mesh [13,36,48], and implicit field [4,28,38,41,48,49]. The voxel decoders [7,61] take advantages of conventional 3D convolution operations to generate volumetric grids.…”

Section: Single-view 3d Object Reconstructionmentioning

confidence: 99%

“…As a complement, we introduce surface-aligned features that capture fine details of surface to facilitate radiance field learning. Given an input image I, we first regress a sparse point cloud S of size 1024 from I using a point set generator G S based on GraphX-convolutions [32]. Then, we feed the generated point cloud to a PointNet++ [42] network to extract pointwise features F S .…”

Section: Feature Extractionmentioning

confidence: 99%

See 1 more Smart Citation

PVSeRF: Joint Pixel-, Voxel- and Surface-Aligned Radiance Field for Single-Image Novel View Synthesis

Yu¹,

Tang²,

Qin³

et al. 2022

Preprint

View full text Add to dashboard Cite

We present PVSeRF, a learning framework that reconstructs neural radiance fields from single-view RGB images, for novel view synthesis. Previous solutions, such as pix-elNeRF [66], rely only on pixel-aligned features and suffer from feature ambiguity issues. As a result, they struggle with the disentanglement of geometry and appearance, leading to implausible geometries and blurry results. To address this challenge, we propose to incorporate explicit geometry reasoning and combine it with pixel-aligned features for radiance field prediction. Specifically, in addition to pixel-aligned features, we further constrain the radiance field learning to be conditioned on i) voxel-aligned features learned from a coarse volumetric grid and ii) fine surfacealigned features extracted from a regressed point cloud. We show that the introduction of such geometry-aware features helps to achieve a better disentanglement between appearance and geometry, i.e. recovering more accurate geometries and synthesizing higher quality images of novel views. Extensive experiments against state-of-the-art methods on ShapeNet benchmarks demonstrate the superiority of our approach for single-image novel view synthesis.

show abstract

Section: Single-view 3d Object Reconstructionmentioning

confidence: 99%

Section: Feature Extractionmentioning

confidence: 99%

PVSeRF: Joint Pixel-, Voxel- and Surface-Aligned Radiance Field for Single-Image Novel View Synthesis

Yu¹,

Tang²,

Qin³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Deep-learning based methods are the most preferred approach for end-to-end inference of 3D geometry from a single image [32]. [33] proposed a convolutional neural network (CNN) to decompose facial images into albedo, shading, and normal maps.…”

Section: Related Workmentioning

confidence: 99%

Local Spherical Harmonics for Facial Shape and Albedo Estimation

et al. 2020

Self Cite

View full text Add to dashboard Cite

In this paper, we present a novel facial albedo and 3D shape recovery method with a local spherical harmonic illumination model. From a face in an image, the proposed method can produce a highquality 3D shape and albedo using a novel parameterization of local illuminations. Because a facial shape is partially convex, a single spherical harmonics is generally used for the illumination of a face within a constrained illumination environment. However, when a facial image is captured in an unconstrained scene, the illumination is inappropriately estimated due to the presence of shadow and specular reflections. To address this issue, we propose a novel local spherical harmonic illumination model for representing the illumination of a face. Unlike the existing parameterization of local illumination, our local spherical harmonic illumination model utilizes a smooth weight function for the seamless representation of natural illumination. Therefore, the albedo and shape information in an image can be precisely estimated using the first-order spherical harmonics. For accurate estimation of albedo, we also utilize facial albedo statistics to prevent the estimated albedo from becoming biased toward input image. Furthermore, we developed an accurate and reliable 3D shape reconstruction method from a normal map based on tetrahedron-based deformation. Comparing to the Laplacian deformation based method, our method is applicable to any mesh regardless of its structure. Through rigorous experiments, we demonstrate that the proposed local spherical harmonic illumination model is effective in estimating the complex illumination and can recover a highquality facial albedo and 3D shape.

show abstract

“…Recent advances in graph convolution networks [14] suggest that graph representations could provide better features for point cloud processing. Such a representation already outperforms the state-of-the-art in many other computer vision tasks [32,19,28,34]. Thus, we investigate the use of a graph representation for LiDAR point cloud in the task of 3D vehicle detection.…”

Section: R-gcn C-gcnmentioning

confidence: 99%

“…GCNs have an increased receptive field with respect to PointNet++ [24] since the edge connections are dynamically assigned for each intermediate layer. With such features, GCNs have shown success in tasks such as segmentation [32], point cloud deformation [19], adversarial generation [28] and point cloud registration [34]. However, To the best of our knowledge, we introduce the first 3D vehicle detection module using a graph representation on LiDAR input.…”

Section: Related Workmentioning

confidence: 99%

PointRGCN: Graph Convolution Networks for 3D Vehicles Detection Refinement

Zarzar,

Giancola,

Ghanem

2019

Preprint

View full text Add to dashboard Cite

In autonomous driving pipelines, perception modules provide a visual understanding of the surrounding road scene. Among the perception tasks, vehicle detection is of paramount importance for a safe driving as it identifies the position of other agents sharing the road. In our work, we propose PointRGCN: a graph-based 3D object detection pipeline based on graph convolutional networks (GCNs) which operates exclusively on 3D LiDAR point clouds. To perform more accurate 3D object detection, we leverage a graph representation that performs proposal feature and context aggregation. We integrate residual GCNs in a twostage 3D object detection pipeline, where 3D object proposals are refined using a novel graph representation. In particular, R-GCN is a residual GCN that classifies and regresses 3D proposals, and C-GCN is a contextual GCN that further refines proposals by sharing contextual information between multiple proposals. We integrate our refinement modules into a novel 3D detection pipeline, PointRGCN, and achieve state-of-the-art performance on the easy difficulty for the bird eye view detection task.

show abstract

GraphX-Convolution for Point Cloud Deformation in 2D-to-3D Conversion

Cited by 33 publications

References 26 publications

PVSeRF: Joint Pixel-, Voxel- and Surface-Aligned Radiance Field for Single-Image Novel View Synthesis

PVSeRF: Joint Pixel-, Voxel- and Surface-Aligned Radiance Field for Single-Image Novel View Synthesis

Local Spherical Harmonics for Facial Shape and Albedo Estimation

PointRGCN: Graph Convolution Networks for 3D Vehicles Detection Refinement

Contact Info

Product

Resources

About