Accelerated Inference in Markov Random Fields via Smooth Riemannian Optimization

Abstract

Markov Random Fields (MRFs) are a popular model for several pattern recognition and reconstruction problems in robotics and computer vision. Inference in MRFs is intractable in general and related work resorts to approximation algorithms. Among those techniques, semidefinite programming (SDP) relaxations have been shown to provide accurate estimates while scaling poorly with the problem size and being typically slow for practical applications. Our first contribution is to design a dual ascent method to solve standard SDP relaxations that takes advantage of the geometric structure of the problem to speed up computation. This technique, named Dual Ascent Riemannian Staircase (DARS), is able to solve large problem instances in seconds. Our second contribution is to develop a second and faster approach. The backbone of this second approach is a novel SDP relaxation combined with a fast and scalable solver based on smooth Riemannian optimization. We show that this approach, named Fast Unconstrained SEmidefinite Solver (FUSES), can solve large problems in milliseconds. Contrarily to local MRF solvers, e.g., loopy belief propagation, our approaches do not require an initial guess. Moreover, we leverage recent results from optimization theory to provide per-instance sub-optimality guarantees. We demonstrate the proposed approaches in multi-class image segmentation problems. Extensive experimental evidence shows that (i) FUSES and DARS produce near-optimal solutions, attaining an objective within 0.1% of the optimum, (ii) FUSES and DARS are remarkably faster than general-purpose SDP solvers, and FUSES is more than two orders of magnitude faster than DARS while attaining similar solution quality, (iii) FUSES is faster than local search methods while being a global solver.

Graduated Non-Convexity for Robust Spatial Perception: From Non-Minimal Solvers to Global Outlier Rejection

“…While the combination of GNC and Black-Rangarajan duality has been investigated in related works [13], [39], its applicability has been limited by the lack of global solvers for the variable update in (9). For instance, [39] focuses on a specific problem (point cloud registration) where (9) can be solved in closed form [4], [3], while [13] focuses on a Markov Random Field formulation for which global solvers and heuristics exist [52]. One of the main insights behind our approach is that modern non-minimal solvers (developed over the last 5 years) allow solving (9) globally for a broader class of problems, including spatial perception problems such as SLAM, mesh registration, and object localization from images.…”

Kimera: From SLAM to spatial perception with 3D dynamic scene graphs

“…Note that the “Office” scene in is the same as in , but the trajectories are different. Finally, to avoid biasing the results towards a particular 2D semantic segmentation method, we use ground-truth 2D semantic segmentations and we refer the reader to Hu and Carlone (2019) for a review of potential alternatives.…”

Kimera: an Open-Source Library for Real-Time Metric-Semantic Localization and Mapping

“…2) Semantic annotation: Kimera-Semantics uses 2D semantically labeled images (produced at each keyframe) to semantically annotate the global mesh; the 2D semantic labels can be obtained using off-the-shelf tools for pixel-level 2D semantic segmentation, e.g., deep neural networks [7]- [9], [64]- [69] or classical MRF-based approaches [70]. To this end, during the bundled raycasting, we also propagate the semantic labels.…”

“…To evaluate the accuracy of the metric-semantic reconstruction from Kimera-Semantics, we use a photo-realistic Unity-based simulator provided by MIT Lincoln Lab, that provides sensor streams (in ROS) and ground truth for both the geometry and the semantics of the scene, and has an interface similar to [20], [21]. To avoid biasing the results towards a particular 2D semantic segmentation method, we use ground truth 2D semantic segmentations and we refer the reader to [70] for potential alternatives.…”