Sublabel-Accurate Convex Relaxation of Vectorial Multilabel Energies

Abstract

Convex relaxations of multilabel problems have been demonstrated to produce provably optimal or near-optimal solutions to a variety of computer vision problems. Yet, they are of limited practical use as they require a fine discretization of the label space, entailing a huge demand in memory and runtime. In this work, we propose the first sublabel accurate convex relaxation for vectorial multilabel problems. Our key idea is to approximate the dataterm in a piecewise convex (rather than piecewise linear) manner. As a result we have a more faithful approximation of the original cost function that provides a meaningful interpretation for fractional solutions of the relaxed convex problem.

Lifting Methods for Manifold-Valued Variational Problems

“…In more recent so-called sublabel-accurate approaches for scalar and vectorial ranges Γ, more emphasis is put on the discretization [71,52,38] to get rid of label bias in models with total variation regularization, which allows to greatly reduce the number of discretizations points for the range Γ. In a recent publication [50], the gain in sublabel accuracy is explained to be caused by an implicit application of first-order finite elements on Γ as opposed to previous approaches that can be interpreted as using zero-order elements, which naturally introduces label-bias.…”

“…Left: The method proposed in [47] does not force the solution to assume values at the grid points (labels), but still shows significant bias towards edges of the grid (blue curve). Second from left: With the same number of labels, the method from [38] is able to reduce label bias by improving data term discretization. Second from right: Furthermore, the method from [38] allows to exploit the convexity of the data term to get decent results with as little as four grid points.…”

“…Left: The method proposed in [47] does not force the solution to take values at the grid points, but still shows significant grid bias. Center: With the same number of labels, our proposed method, motivated by [38], reduces label bias by improving data term discretization. Right: Furthermore, our method can get excellent results with as little as 6 grid points (right).…”

“…Left: In the lifting approach [47] for manifold-valued problems, the data term is interpolated linearly between the labels. Right: Based on ideas from recent scalar and vectorial lifting approaches [52,38], we interpolate piecewise convex between the labels. Fig.…”

“…This agrees with our setting if z → ρ(x, ι(z)) is affine linear on each simplex T ∈ T h , as then q(x, z) = ρ(x, ι(z)) maximizes the objective function for any p and v. Hence, the model in [47] doesn't take into account any information about ρ below the resolution of the triangulation. We improve this by implementing the epigraph constraints ρ * T (q T,1 (x)) ≤ b T (x) as proposed in [38] using a convex approximation of ρ T (see Fig. 8).…”