Curvature Prior for MRF-Based Segmentation and Shape Inpainting

Abstract

Most image labeling problems such as segmentation and image reconstruction are fundamentally ill-posed and suffer from ambiguities and noise. Higher order image priors encode high level structural dependencies between pixels and are key to overcoming these problems. However, these priors in general lead to computationally intractable models. This paper addresses the problem of discovering compact representations of higher order priors which allow efficient inference. We propose a framework for solving this problem which uses a recently proposed representation of higher order functions where they are encoded as lower envelopes of linear functions. Maximum a Posterior inference on our learned models reduces to minimizing a pairwise function of discrete variables, which can be done approximately using standard methods. Although this is a primarily theoretical paper, we also demonstrate the practical effectiveness of our framework on the problem of learning a shape prior for image segmentation and reconstruction. We show that our framework can learn a compact representation that approximates a prior that encourages low curvature shapes. We evaluate the approximation accuracy, discuss properties of the trained model, and show various results for shape inpainting and image segmentation.

Partial Enumeration and Curvature Regularization

“…The mathematical justification of this approach to curvature estimation is not fully explained and several presented plots indicate its limited accuracy. As stated in [28], "the plots do also reveal the fact that we consistently overestimate the true curvature cost." The extreme "hard" case of this method may reduce to our technique if the cost of each pattern is assigned according to our integral geometry equations in Fig.1(c).…”

“…Grid patches were also recently used for curvature evaluation in [28]. Unlike our integral geometry in Fig.1(c), their method computes a minimum response over a number of affine filters encoding some learned "soft" patterns.…”

“…The extreme "hard" case of this method may reduce to our technique if the cost of each pattern is assigned according to our integral geometry equations in Fig.1(c). However, this case makes redundant the filter response minimization and the pattern costs learning, which are the key technical ideas in [28].…”

Piecewise‐Planar 3D Reconstruction with Edge and Corner Regularization

“…Therefore, our method ignores the problem of obtaining a good triangulation of each planar face and concentrate on the actual surface estimation problem. Our approach bares some similarity with some methods used for 2D image segmentation that also rely on higher‐order regularization terms such as region boundary curvature [SKC09, SKR12]. In [SK11], a 3D surface is completed based on higher‐order priors using a global binary labelization of a volume partition, similarly to us.…”

Max-Margin Boltzmann Machines for Object Segmentation

“…The prediction from the observations x to the labels y is usually realized in the conditional models p(y|x), i.e., Conditional Random Fields (CRFs) [15], which allow flexible use of various long-range features from observations x. Pairwise potentials [22], although admitting efficient inference, can only capture limited local structure, such as smoothness and edges. High-order potentials are able to capture long-range interactions between pixel labels through bottom-up segmentation [13], pattern-based priors [19,21]. Beyond the generic high-order priors, the ObjCut algorithm [14] introduces category-specific object models into MRFs and has shown good segmentation performance on articulated objects.…”