Combinatorial preconditioners and multilevel solvers for problems in computer vision and image processing

Abstract

Linear systems and eigen-calculations on symmetric diagonally dominant matrices (SDDs) occur ubiquitously in computer vision, computer graphics, and machine learning. In the past decade a multitude of specialized solvers have been developed to tackle restricted instances of SDD systems for a diverse collection of problems including segmentation, gradient inpainting and total variation. In this paper we explain and apply the support theory of graphs, a set of of techniques developed by the computer science theory community, to construct SDD solvers with provable properties. To demonstrate the power of these techniques, we describe an efficient multigrid-like solver which is based on support theory principles. The solver tackles problems in fairly general and arbitrarily weighted topologies not supported by prior solvers. It achieves state of the art empirical results while providing robust guarantees on the speed of convergence. The method is evaluated on a variety of vision applications.

A Nearly-m log n Time Solver for SDD Linear Systems

“…Solvers for symmetric diagonally dominant (SDD) 3 systems are a crucial component of the fastest known algorithms for a multitude of problems that include (i) Computing the first non-trivial (Fiedler) eigenvector of the graph, with well known applications to the sparsest-cut problem [1], [2], [3]; (ii) Generating spectral sparsifiers that also act as cut-preserving sparsifiers [4]; (iii) Solving linear systems derived from elliptic finite element discretizations of a significant class of partial differential equations [5]; (iv) Generalized lossy flow problems [6]; (v) Generating random spanning trees [7]; (vi) Faster maximum flow algorithms [8]; and (vii) Several optimization problems in computer vision [9], [10] and graphics [11], [12].…”

Multigrid and multilevel preconditioners for computational photography

“…Algebraic multigrid techniques also use an adaptive subsampling of variables to define the coarselevel grid. Combinatorial multigrid (CMG) techniques resemble algebraic multigrid but use an agglomerative coarsening scheme in which clusters of fine-level variables get replaced by a single coarse-level variable [Koutis et al 2009]. This has the advantage of simpler and faster interpolation and also supports deriving bounds on the growth in relative condition numbers as the number of levels increases.…”

“…Originally developed for homogeneous elliptic differential equations, MG methods now constitute a family of methods under a common framework that can be used to solve both inhomogeneous elliptic and non-elliptic differential equations. This family includes algebraic multigrid (AMG) [Briggs et al 2000;Trottenberg et al 2000;Kushnir et al 2010;Napov and Notay 2011] and combinatorial multigrid (CMG) techniques [Koutis et al 2009]. …”

“…While the locally adaptive hierarchical basis function technique developed by Szeliski [Szeliski 2006] showed impressive speedups over earlier non-adaptive basis functions [Szeliski 1990], it was never adequately compared to state-of-the art multigrid techniques such as algebraic multigrid [Briggs et al 2000;Trottenberg et al 2000] or to newer techniques such as combinatorial multigrid [Koutis et al 2009]. Furthermore, the original technique was restricted to problems defined on four neighbor (N4) grids.…”

Spectral Segmentation Using Cartoon-Texture Decomposition and Inner Product-Based Metric

“…Another important issue in the context of spectral cuts is the computational cost. Computing the eigenstructure of a graph is a very time consuming task, hampering the direct use of spectral cuts in high resolution images [15].…”