Occlusion Boundaries from Motion: Low-Level Detection and Mid-Level Reasoning

Abstract

The boundaries of objects in an image are often considered a nuisance to be "handled" due to the occlusion they exhibit. Since most, if not all, computer vision techniques aggregate information spatially within a scene, information spanning these boundaries, and therefore from different physical surfaces, is invariably and erroneously considered together. In addition, these boundaries convey important perceptual information about 3D scene structure and shape. Consequently, their identification can benefit many different computer vision pursuits, from low-level processing techniques to high-level reasoning tasks.While much focus in computer vision is placed on the processing of individual, static images, many applications actually offer video, or sequences of images, as input. The extra temporal dimension of the data allows the motion of the camera or the scene to be used in processing. In this paper, we focus on the exploitation of subtle relative-motion cues present at occlusion boundaries. When combined with more standard appearance information, we demonstrate these cues' utility in detecting occlusion boundaries locally. We also present a novel, mid-level model for reasoning more globally about object boundaries and propagating such local information to extract improved, extended boundaries.

Efficient Closed-Form Solution to Generalized Boundary Detection

“…Second, we perform experiments on occlusion boundary detection in short video clips. Multiple frames, closely spaced in time, provide significantly more information about dynamic scenes and make occlusion boundary detection possible, as shown in recent work [6][7][8]24]. Third, we experiment with RGB-D video frames and show that depth can be effectively combined with color and optical flow to detect moving occlusion boundaries.…”

“…This is in contrast to current approaches [6,7,9] that process low and mid-level layers separately and combine them in different ways to detect different types of boundaries. 4) we only learn a small set of parameters, enabling efficient training with limited data.…”

“…State-of-the-art techniques for occlusion boundary detection in video are based on combining, in various ways, the outputs of existing boundary detectors for static color images with optical flow, followed by a global processing phase [6][7][8]24]. Table 3 compares Gb against reported results on the CMU Motion Dataset [6] We use, as one of our layers, the flow computed using Sun et al's public code [27].…”

“…Table 3 compares Gb against reported results on the CMU Motion Dataset [6] We use, as one of our layers, the flow computed using Sun et al's public code [27]. Additionally, Gb uses color and soft segmentation (Section 4).…”

“…The majority of papers on edge detection have focused on using only low-level cues, such as pixel intensity or color [1][2][3][4][5]. Recent work has started exploring the problem of boundary detection based on higher-level representations of the image, such as motion, surface and depth cues [6][7][8], segmentation [9], as well as category specific information [10,11].…”