Fully Convolutional Geometric Features for Category-level Object Alignment

Feng, Qiaojun; Atanasov, Nikolay

doi:10.1109/iros45743.2020.9341550

“…Therefore, after testing compatibility between every pair of keypoints, we can find inliers by searching for the largest set of mutually compatible measurements. References [20,45,65,87] have already established that the largest set of mutually compatible measurements can be found by computing the maximum clique of a graph where nodes correspond to the 3D keypoints and an edge connects nodes i and j is the corresponding measurements satisfy the compatibility test (22). While we refer the reader to those papers for details, here we observe that such graphtheoretic approach has been shown to remove a large amount of gross outliers [87] (while preserving all inliers).…”

Section: A Outlier Pruning For Category-level Perceptionmentioning

confidence: 68%

“…Such approaches first recover the position of semantic keypoints [56] in the images with neural networks, and then recover the 3D pose of the object by solving a geometric optimization problem [31,53,56,57,64]. In some works, a canonical coordinate space is predicted by a network instead of relying on geometric reasoning [14,22,41,78]. Lim et al [42] establish 2D-3D correspondences between images and textureless CAD models by using HOG descriptors, and render edgemaps of the CAD models.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Optimal Pose and Shape Estimation for Category-level 3D Object Perception

Shi

¹

,

Yang

²

,

Carlone

³

2021

Robotics: Science and Systems XVII

View full text Add to dashboard Cite

We consider a category-level perception problem, where one is given 3D sensor data picturing an object of a given category (e.g., a car), and has to reconstruct the pose and shape of the object despite intra-class variability (i.e., different car models have different shapes). We consider an active shape model, where -for an object category-we are given a library of potential CAD models describing objects in that category, and we adopt a standard formulation where pose and shape estimation are formulated as a non-convex optimization. Our first contribution is to provide the first certifiably optimal solver for pose and shape estimation. In particular, we show that rotation estimation can be decoupled from the estimation of the object translation and shape, and we demonstrate that (i) the optimal object rotation can be computed via a tight (small-size) semidefinite relaxation, and (ii) the translation and shape parameters can be computed in closed-form given the rotation. Our second contribution is to add an outlier rejection layer to our solver, hence making it robust to a large number of misdetections. Towards this goal, we wrap our optimal solver in a robust estimation scheme based on graduated non-convexity. To further enhance robustness to outliers, we also develop the first graph-theoretic formulation to prune outliers in category-level perception, which removes outliers via convex hull and maximum clique computations; the resulting approach is robust to 70 − 90% outliers. Our third contribution is an extensive experimental evaluation. Besides providing an ablation study on a simulated dataset and on the PASCAL3D+ dataset, we combine our solver with a deep-learned keypoint detector, and show that the resulting approach improves over the state of the art in vehicle pose estimation in the ApolloScape datasets.

show abstract

“…Therefore, after testing compatibility between every pair of keypoints, we can find inliers by searching for the largest set of mutually compatible measurements. References [20,45,64,86] have already established that the largest set of mutually compatible measurements can be found by computing the maximum clique of a graph where nodes correspond to the 3D keypoints and an edge connects nodes i and j is the corresponding measurements satisfy the compatibility test (22). While we refer the reader to those papers for details, here we observe that such graphtheoretic approach has been shown to remove a large amount of gross outliers [86] (while preserving all inliers).…”

Section: A Outlier Pruning For Category-level Perceptionmentioning

confidence: 69%

“…Such approaches first recover the position of semantic keypoints [55] in the images with neural networks, and then recover the 3D pose of the object by solving a geometric optimization problem [31,52,55,56,63]. In some works, a canonical coordinate space is predicted by a network instead of relying on geometric reasoning [14,22,41,77]. Lim et al [42] establish 2D-3D correspondences between images and textureless CAD models by using HOG descriptors, and render edgemaps of the CAD models.…”

Section: Related Workmentioning

confidence: 99%

Optimal Pose and Shape Estimation for Category-level 3D Object Perception

Shi

¹

,

Yang

²

,

Carlone

³

2021

Preprint

0

View full text Add to dashboard Cite

We consider a category-level perception problem, where one is given 3D sensor data picturing an object of a given category (e.g., a car), and has to reconstruct the pose and shape of the object despite intra-class variability (i.e., different car models have different shapes). We consider an active shape model, where -for an object category-we are given a library of potential CAD models describing objects in that category, and we adopt a standard formulation where pose and shape estimation are formulated as a non-convex optimization. Our first contribution is to provide the first certifiably optimal solver for pose and shape estimation. In particular, we show that rotation estimation can be decoupled from the estimation of the object translation and shape, and we demonstrate that (i) the optimal object rotation can be computed via a tight (small-size) semidefinite relaxation, and (ii) the translation and shape parameters can be computed in closed-form given the rotation. Our second contribution is to add an outlier rejection layer to our solver, hence making it robust to a large number of misdetections. Towards this goal, we wrap our optimal solver in a robust estimation scheme based on graduated non-convexity. To further enhance robustness to outliers, we also develop the first graph-theoretic formulation to prune outliers in category-level perception, which removes outliers via convex hull and maximum clique computations; the resulting approach is robust to 70 − 90% outliers. Our third contribution is an extensive experimental evaluation. Besides providing an ablation study on a simulated dataset and on the PASCAL3D+ dataset, we combine our solver with a deep-learned keypoint detector, and show that the resulting approach improves over the state of the art in vehicle pose estimation in the ApolloScape datasets.

show abstract