Learning Spatial Context: Using Stuff to Find Things

Abstract

Abstract. The sliding window approach of detecting rigid objects (such as cars) is predicated on the belief that the object can be identified from the appearance in a small region around the object. Other types of objects of amorphous spatial extent (e.g., trees, sky), however, are more naturally classified based on texture or color. In this paper, we seek to combine recognition of these two types of objects into a system that leverages "context" toward improving detection. In particular, we cluster image regions based on their ability to serve as context for the detection of objects. Rather than providing an explicit training set with region labels, our method automatically groups regions based on both their appearance and their relationships to the detections in the image. We show that our things and stuff (TAS) context model produces meaningful clusters that are readily interpretable, and helps improve our detection ability over state-of-the-art detectors. We also present a method for learning the active set of relationships for a particular dataset. We present results on object detection in images from the PASCAL VOC 2005/2006 datasets and on the task of overhead car detection in satellite images, demonstrating significant improvements over state-of-the-art detectors.

SuperParsing: Scalable Nonparametric Image Parsing with Superpixels

“…The literature contains diverse proposed image parsing methods, including ones that estimate labels pixel by pixel [1,2], ones that aggregate features over segmentation regions [3][4][5][6], and ones that predict object bounding boxes [7][8][9][10]. Most of these methods operate with a few pre-defined classes and require a generative or discriminative model to be trained in advance for each class (and sometimes even for each training exemplar [5]).…”

“…The current consensus among recognition researchers is that image parsing requires context (see, e.g., [3,4,9,10]). However, learning and inference with most existing contextual models is slow and non-exact.…”

CollageParsing: Nonparametric Scene Parsing by Adaptive Overlapping Windows

“…Heitz and Koller [16] developed a graphical model to improve the detection of objects ("things") by making use of local context. In this work, local context refers to "stuff" classes such as road or sky.…”

“…Scene parsing is the challenging problem of assigning a semantic label to every pixel in the image. Semantic labels can span both amorphous background categories such as grass or sea (sometimes referred to "stuff" in the literature [16]), as well as localized object categories such as person or car (sometimes referred to as "things").…”

Modeling and Inference with Relational Dynamic Bayesian Networks

“…These algorithms express the knowledge about the scene around the objects with a probability distribution and use it to accomplish the task of object recognition given their context. In (Heitz & Koller, 2008), for example, a system that leverages "context" toward improving detection is presented. The method clusters image regions based on their ability to serve as context for the detection of objects.…”