Perceptual Organization and Recognition of Indoor Scenes from RGB-D Images

Abstract

We address the problems of contour detection, bottomup grouping and semantic segmentation using RGB-D data. We focus on the challenging setting of cluttered indoor scenes, and evaluate our approach on the recently introduced NYU-Depth V2 (NYUD2) dataset [27]. We propose algorithms for object boundary detection and hierarchical segmentation that generalize the gP b − ucm approach of [2] by making effective use of depth information. We show that our system can label each contour with its type (depth, normal or albedo). We also propose a generic method for long-range amodal completion of surfaces and show its effectiveness in grouping. We then turn to the problem of semantic segmentation and propose a simple approach that classifies superpixels into the 40 dominant object categories in NYUD2. We use both generic and class-specific features to encode the appearance and geometry of objects. We also show how our approach can be used for scene classification, and how this contextual information in turn improves object recognition. In all of these tasks, we report significant improvements over the state-of-the-art.

Instance Segmentation of Indoor Scenes Using a Coverage Loss

“…We additionally experiment with a superset of region features from [30] and [5] as well as compare the convolutional network features to Sparse Coded SIFT features from [31]. Our pairwise region features are a superset of pairwise region and boundary features from [30] [5].…”

Depth map compression via 3D region-based representation

“…Results generated with the proposed scheme are compared against the segmentation produced with rgbd-ucm [12] and with ucm using only the depth image. For rgbd-ucm, a hierarchical segmentation is generated using color and depth clues.…”

“…This inter-view redundancy can be removed by extracting the geometrical structure of the scene. 3D representations have been widely used in multiple applications, from object segmentation to scene recognition [3,11,12]. In [17], a multiview object co-segmentation method is proposed that estimates a depth map with a set of 3D planar surfaces.…”

FPM: Fine Pose Parts-Based Model with 3D CAD Models

“…In order to better understand and interact with our environment, it is important to tackle the problem of fine pose estimation as shown in Figure 1 While fine pose estimation for simple 3D shapes has been studied since the early days of computer vision [28], estimating the fine pose of articulate 3D objects has received little attention. Given the recent success of the computer vision community in addressing object detection [8,13] and 3D scene understanding [12,24,4,33,16], fine pose estimation is becoming more and more approachable. In this work, we tackle the problem of instance-level fine pose estimation given only a single-view image.…”