Monocular Depth Ordering Using T-Junctions and Convexity Occlusion Cues

Abstract

Abstract-This paper proposes a system that relates objects in an image using occlusion cues and arranges them according to depth. The system does not rely on a priori knowledge of the scene structure and focuses on detecting special points, such as T-junctions and highly convex contours, to infer the depth relationships between objects in the scene. The system makes extensive use of the binary partition tree as hierarchical region-based image representation jointly with a new approach for candidate T-junction estimation. Since some regions may not involve T-junctions, occlusion is also detected by examining convex shapes on region boundaries. Combining T-junctions and convexity leads to a system which only relies on low level depth cues and does not rely on semantic information. However, it shows a similar or better performance with the state-of-the-art while not assuming any type of scene.As an extension of the automatic depth ordering system, a semi-automatic approach is also proposed. If the user provides the depth order for a subset of regions in the image, the system is able to easily integrate this user information to the final depth order for the complete image. For some applications, user interaction can naturally be integrated, improving the quality of the automatically generated depth map.

Precision-Recall-Classification Evaluation Framework: Application to Depth Estimation on Single Images

“…Their performances as given in their respective papers are reported in Table 1. The LDC and GDC measures are reported for the BPT+TJC and UCM+TJC approaches of [17], the angular embedding (AE) of [11], the occlusion boundary detection (OB) of [6], the learning depth based (LD) approach of [20], the PO approach of [2] and the proposed algorithm using the region PO-based graph cut on BPT. …”

“…BPT+TJC [17] UCM+TJC [17] Other approaches rely on higher level features such as surface orientation or semantics. In this research line, the work of [20] oversegments the image and infers depth maps using a random field.…”

“…Table 1 reports the performance of several methods: [17], [9] and [18]. The usual way to measure the performances is to decouple segmentation from depth classification by providing two independent measures.…”

“…Nevertheless, low level cues do offer a good starting point to determine depth order. Works such as [3,17] attempt to estimate the depth order through the explicit detection of occlusion cues and rely on two perceptual ideas: 1) convex regions appear to be occluding and 2) in case of T-junctions, the region forming the largest angle is the occluding region. [11] proposes an extension of the gP b algorithm [12] to provide a figure/ground order based on the convexity of detected contours.…”

“…The novelty of this approach is that it does not have to explicitly deal with cue detection but occlusion arises naturally from the image model. [17,9,18] using figure/ground classification accuracy. All measures are extracted from the respective papers.…”