Learning and Matching Multi-View Descriptors for Registration of Point Clouds

Zhou, Lei; Zhu, Siyu; Luo, Zixin; Shen, Tianwei; Zhen, Mingmin; Fang, Tian; Quan, Long

doi:10.1007/978-3-030-01267-0_31

Cited by 45 publications

(27 citation statements)

References 64 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This is also a difficulty faced by many tasks on graphs [42]. Instead of defining heuristic neighbors for correspondences as done in previous works [15,45], we exploit Differentiable Pooling [42] to cluster nodes in a learnable manner and capture the local context. However, the original DiffPool Network is not applicable in our case because it does not give a full size prediction.…”

Section: Geometric Deep Learningmentioning

confidence: 99%

“…One of the challenges in mitigating the above limitations is exploiting neighbors to encoding local context. Unlike 3D point clouds, sparse matches have no well-defined neighbors, where this issue is previously tackled in bilateral domain [15] (2D spatial domain and 2D motion domain) or by a graphical model [45]. Besides, another challenge is modeling the relation between correspondences since they are unordered and have no stable relations to be captured.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Learning Two-View Correspondences and Geometry Using Order-Aware Network

Zhang

Sun²,

Luo

et al. 2019

2019 IEEE/CVF International Conference on Computer Vision (ICCV)

Self Cite

286

113

View full text Add to dashboard Cite

Establishing correspondences between two images requires both local and global spatial context. Given putative correspondences of feature points in two views, in this paper, we propose Order-Aware Network, which infers the probabilities of correspondences being inliers and regresses the relative pose encoded by the essential matrix. Specifically, this proposed network is built hierarchically and comprises three novel operations. First, to capture the local context of sparse correspondences, the network clusters unordered input correspondences by learning a soft assignment matrix. These clusters are in a canonical order and invariant to input permutations. Next, the clusters are spatially correlated to form the global context of correspondences. After that, the context-encoded clusters are recovered back to the original size through a proposed upsampling operator. We intensively experiment on both outdoor and indoor datasets. The accuracy of the twoview geometry and correspondences are significantly improved over the state-of-the-arts. Code will be available at https://github.com/zjhthu/OANet.git.

show abstract

Section: Geometric Deep Learningmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Learning Two-View Correspondences and Geometry Using Order-Aware Network

Zhang

Sun²,

Luo

et al. 2019

2019 IEEE/CVF International Conference on Computer Vision (ICCV)

Self Cite

286

113

View full text Add to dashboard Cite

show abstract

“…Such problems become more obvious in 3D point cloud registration since the description ability of 3D descriptors is generally weaker than those in 2D domain [42,6,44,43,45] due to the irregular density and the lack of useful texture [11]. Thus, geometric consistency, such as length constraint under rigid transformation, becomes important and is commonly utilized by traditional outlier rejection algorithms and analyzed through spectral techniques [38,20], voting schemes [26,74,61], maximum clique [54,12,64], random walk [14], belief propagation [81] or game theory [59]. Meanwhile, some algorithms based on BnB [11] or SDP [37] are accurate but usually have high time complexity.…”

Section: Related Workmentioning

confidence: 99%

PointDSC: Robust Point Cloud Registration using Deep Spatial Consistency

Bai¹,

Luo²,

Zhou³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

Removing outlier correspondences is one of the critical steps for successful feature-based point cloud registration. Despite the increasing popularity of introducing deep learning techniques in this field, spatial consistency, which is essentially established by a Euclidean transformation between point clouds, has received almost no individual attention in existing learning frameworks. In this paper, we present PointDSC, a novel deep neural network that explicitly incorporates spatial consistency for pruning outlier correspondences. First, we propose a nonlocal feature aggregation module, weighted by both feature and spatial coherence, for feature embedding of the input correspondences. Second, we formulate a differentiable spectral matching module, supervised by pairwise spatial compatibility, to estimate the inlier confidence of each correspondence from the embedded features. With modest computation cost, our method outperforms the state-of-the-art handcrafted and learning-based outlier rejection approaches on several real-world datasets by a significant margin. We also show its wide applicability by combining PointDSC with different 3D local descriptors.[code release]

show abstract

“…Huang et al [13] learn local features from 3D multiview projected images and fuse multiview features into local shape descriptors. Zhou et al [14] conduct similar work and create multiview fused features. Burges et al [15] train CNNs to map high-dimensional features into a lowdimensional embedding space and make features more discriminative.…”

Section: A Local Feature Extraction For 3d Shapesmentioning

confidence: 99%

3D Patch-Based Sparse Learning for Style Feature Extraction

Pan

Liu

2019

IEEE Access

View full text Add to dashboard Cite

How to extract features of different shapes for style similarity evaluation is a very challenging research topic. Different from methods based on predefined style templates, we perform sparse learning and triplet embedding directly from the 3D shape local descriptors. The proposed method can adaptively generate element-level style features of different 3D patches for style similarity evaluation. The proposed algorithm mainly consists of the following steps. First, given a heterogeneous 3D shape collection, we learn mid-level style features by Siamese networks based on curvature-guided sampling directly on 3D shapes, which is a view-independent and part-independent approach. Second, we perform sparse coding to reduce the dimension of mid-level local features and achieve more compact and discriminative style features, named element-level features. Finally, we enhance our element-level style features by triplet embedding learning over crowdsourced triplets, which maps similar examples close to each other and dissimilar examples farther apart. The proposed algorithm can aggregate different local features into a global style feature representation and shows satisfactory discriminability. Our method can avoid the loss of style features from the 3D model to the views and can learn more discriminative style features by sparse learning. The experimental results show that our sparse style feature representation significantly improves the accuracy compared with recent state-of-the-art algorithms. INDEX TERMS 3D shape analysis, style feature, sparse learning, 3D patch.

show abstract

Learning and Matching Multi-View Descriptors for Registration of Point Clouds

Cited by 45 publications

References 64 publications

Learning Two-View Correspondences and Geometry Using Order-Aware Network

Learning Two-View Correspondences and Geometry Using Order-Aware Network

PointDSC: Robust Point Cloud Registration using Deep Spatial Consistency

3D Patch-Based Sparse Learning for Style Feature Extraction

Contact Info

Product

Resources

About