Shared Sampling for Real‐Time Alpha Matting

Abstract

Image matting aims at extracting foreground elements from an image by means of color and opacity (alpha) estimation. While a lot of progress has been made in recent years on improving the accuracy of matting techniques, one common problem persisted: the low speed of matte computation. We present the first real-time matting technique for natural images and videos. Our technique is based on the observation that, for small neighborhoods, pixels tend to share similar attributes. Therefore, independently treating each pixel in the unknown regions of a trimap results in a lot of redundant work. We show how this computation can be significantly and safely reduced by means of a careful selection of pairs of background and foreground samples. Our technique achieves speedups of up to two orders of magnitude compared to previous ones, while producing high-quality alpha mattes. The quality of our results has been verified through an independent benchmark. The speed of our technique enables, for the first time, real-time alpha matting of videos, and has the potential to enable a new class of exciting applications.

Confidence-driven image co-matting

“…Essentially, the matting confidence directly correlates with the accuracy of the computed alpha matte. Notice that the definition of confidence here differs from that in previous sampling based approaches [2,4,5]. The latter is used to indicate how well the selected color samples satisfy the linear constraint in Eqn.…”

“…For remaining pixels, their corresponding weights are assigned as the color sampling confidence as described in [5], denoted as f s i in Eqn. 4. By doing this, we ensure that high confidence alpha values are kept unchanged while lower confidence ones are refined in the optimization process.…”

“…Hybrid methods [2,3,4,5] combine the above two approaches, assuming that sampling errors can be eliminated under smoothness constraints, while affinitybased propagation error can be suppressed by accurate color sampling. The recent work by Chen et al [13] integrates sampling-based priors with both local and nonlocal affinities together to achieve the best performance on the benchmark dataset [14].…”

“…Rhemann et al [3] collect nearby samples in geodesic distance instead of Euclidean distance. Shared matting [4] shoots rays in several directions and samples known boundary pixels on the rays. Best sample pairs are further propagated among neighboring pixels.…”

Practical automatic background substitution for live video

“…It no longer requires a known background and static camera, and takes a user drawn trimap or strokes to tell if a pixel belongs to the foreground/background/unknown region. For images, previous methods are often samplingbased [30], affinity-based [24], or a combination of both [31], computing alpha values for the unknown region based on the known region information. For video, Chuang et al [15] use optical flow to propagate the trimap from one frame to another.…”