Bayesian geometric modeling of indoor scenes

Abstract

We propose a method for understanding the 3D geometry of indoor environments (e.g. bedrooms, kitchens) while simultaneously identifying objects in the scene (e.g. beds, couches, doors). We focus on how modeling the geometry and location of specific objects is helpful for indoor scene understanding. For example, beds are shorter than they are wide, and are more likely to be in the center of the room than cabinets, which are tall and narrow. We use a generative statistical model that integrates a camera model, an enclosing room "box", frames (windows, doors, pictures), and objects (beds, tables, couches, cabinets), each with their own prior on size, relative dimensions, and locations. We fit the parameters of this complex, multi-dimensional statistical model using an MCMC sampling approach that combines discrete changes (e.g, adding a bed), and continuous parameter changes (e.g., making the bed larger). We find that introducing object category leads to state-of-theart performance on room layout estimation, while also enabling recognition based only on geometry.

Unfolding an Indoor Origami World

“…This sparked a renaissance during which progress started being made on a variety of long-standing 3D understanding problems. In the indoor world, a great deal of effort went into developing constrained models for the prediction of room layout [10] as well as features [6,23,27] and effective methods for inference [4,22,31,32]. While these high-level constraints have been enormously successful in constrained domains (e.g., less cluttered scenes with visible floors such as the datasets of [10,38]), they have not been successfully demonstrated on highly cluttered scenes such as the NYU v2 Depth Dataset [33].…”

Indoor frame recovering via line segments refinement and voting

“…1). The recovered frame could have applications in indoor scenes, i.e., robot navigation, object recognition, 3D reconstruction, and event detection [1,2,3,4]. However, indoor frame recovery remains a challenging task because of illumination variations, weak boundaries, and partial occlusions.…”

“…Orientation map-based algorithms often use the orientation map as a supplemental descriptor for texture descriptors because it can describe the local category of regions. For instance, Schwing et al [13] and Pero et al [4] introduced the orientation map as well as texture for frame inference. However, approaches based on the orientation map often do not work well because such map is derived from detected line segments, which are sensitive to weak boundaries.…”

Data-Driven Scene Understanding from 3D Models

“…More recently, attention has been focused on estimating the locations of objects in these environments [17,24,28,33], and on estimating a scene's free space [13]. New work [18,29] aims to recover free space by localizing cuboids representing object categories and sizes using parametric models as their prior. In contrast, we recover more detailed object geometries and we use non-parametric priors that can capture complex interactions between objects.…”