Context models and out-of-context objects

Abstract

The context of an image encapsulates rich information about how natural scenes and objects are related to each other. Such contextual information has the potential to enable a coherent understanding of natural scenes and images. However, context models have been evaluated mostly based on the improvement of object recognition performance even though it is only one of many ways to exploit contextual information. In this paper, we present a new scene understanding problem for evaluating and applying context models. We are interested in finding scenes and objects that are "out-of-context". Detecting "out-of-context" objects and scenes is challenging because context violations can be detected only if the relationships between objects are carefully and precisely modeled. To address this problem, we evaluate different sources of context information, and present a graphical model that combines these sources. We show that physical support relationships between objects can provide useful contextual information for both object recognition and out-of-context detection.

Recognizing object affordances in terms of spatio-temporal object-object relationships

“…In [19], functional categories of image regions, such as roads and walkways, are segmented and clustered based on the behaviors of moving objects in the vicinity. [6], [7], [20] exploit the contextual associations to recognize objects, however this classification is only performed in terms of appearance without any reasoning about function or affordances. We here go one step further by directly observing object relations rather than constraining individual object observations with contextual information about the relation with other objects.…”

Abnormal Object Detection by Canonical Scene-Based Contextual Model

“…In [1], Choi represents the relation among objects in a tree model for computational efficiency, taking only a few significant relationship among the objects into consideration. This tree model is more expanded in [5] to also encode "support" context, a geometric context for prohibiting floating objects. The strategy of [3] is to first find easy objects, and then search for difficult objects based on the locations of the easy objects.…”

“…Likewise, a "person" who is taller than common adults is also annotated. Even though an abnormal dataset has already been established [1,5], the set is unsuitable for a thorough evaluation of all three types of abnormal objects because the set has not been separated into each type.…”

“…We choose a hybrid model of co-occurrence contextual (CO) and support contextual (SUP) model [5] as the baseline method for abnormal object detection, because our proposed model simultaneously detects abnormal objects that violate co-occurrence context and position/scale context. The number of objects used in the baseline is 10+2 objects, including support objects such as "floor" and "road."…”

“…Whether the proposed abnormal object detection methods are adequate for finding each type of abnormal object must be verified. At present, only a few studies on abnormal object detections have been conducted [1,5]. This paper is inspired by these studies.…”