Filter-Based Mean-Field Inference for Random Fields with Higher-Order Terms and Product Label-Spaces

Abstract

Abstract. Recently, a number of cross bilateral filtering methods have been proposed for solving multi-label problems in computer vision, such as stereo, optical flow and object class segmentation that show an order of magnitude improvement in speed over previous methods. These methods have achieved good results despite using models with only unary and/or pairwise terms. However, previous work has shown the value of using models with higher-order terms e.g. to represent label consistency over large regions, or global co-occurrence relations. We show how these higher-order terms can be formulated such that filter-based inference remains possible. We demonstrate our techniques on joint stereo and object labeling problems, as well as object class segmentation, showing in addition for joint object-stereo labeling how our method provides an efficient approach to inference in product label-spaces. We show that we are able to speed up inference in these models around 10-30 times with respect to competing graph-cut/move-making methods, as well as maintaining or improving accuracy in all cases. We show results on PascalVOC-10 for object class segmentation, and Leuven for joint object-stereo labeling.

PoseField: An Efficient Mean-Field Based Method for Joint Estimation of Human Pose, Segmentation, and Depth

“…This model was later extended by Vineet et.al. [16] to incorporate higher order potentials, and to solve jointly the object-stereo correspondence problems. Let us consider a general form of energy function:…”

“…One such method that can be applied here is mean-field inference [15]. For a certain class of pairwise terms, mean-field inference has been shown to be very powerful in solving the object class segmentation problem, and object-stereo correspondence problems in CRF frameworks, providing an order-of-magnitude speedup [16]. In this paper, we propose a highly efficient filter-based mean-field approach to perform joint estimation of human segmentation, pose, per-pixel part labels, and disparity in the product label space, producing a significant improvement in speed.…”

Learning to approximate global shape priors for figure-ground segmentation

“…Several works appeared to deal with pattern potentials [12,17,26,34]. These potentials can act on rather large sets of variables, but they give the same (high) energy value to most labellings.…”

“…A very popular such class are pairwise functions, involving only terms depending either on one or two variables [6,15,16]. The optimization of higher-order functions is still an open, highly challenging problem [2,17,19,23,26,34].…”

“…A recent trend is to design energies containing higher-order potentials of a particular form, along with an optimizer specialized for that form [2,11,13,17,23,26,34]. The downside of this approach is that different optimizers needs to be invented for each form of higher-order potential, which requires great skill and knowledge.…”