Characterization of 3-D Volumetric Probabilistic Scenes for Object Recognition

Abstract

This paper presents a new volumetric representation for categorizing objects in large-scale 3-D scenes reconstructed from image sequences. This work uses a probabilistic volumetric model (PVM) that combines the ideas of background modeling and volumetric multi-view reconstruction to handle the uncertainty inherent in the problem of reconstructing 3-D structures from 2-D images. The advantages of probabilistic modeling have been demonstrated by recent application of the PVM representation to video image registration, change detection and classification of changes based on PVM context. The applications just mentioned, operate on 2-D projections of the PVM. This paper presents the first work to characterize and use the local 3-D information in the scenes. Two approaches to local feature description are proposed and compared: 1) features derived from a PCA analysis of model neighborhoods; and 2) features derived from the coefficients of a 3-D Taylor series expansion within each neighborhood. The resulting description is used in a bag-of-features approach to classify buildings, houses, cars, planes, and parking lots learned from aerial imagery collected over Providence, RI. It is shown that both feature descriptions explain the data with similar accuracy and their effectiveness for dense-feature categorization is compared for the different classes. Finally, 3-D extensions of the Harris corner detector and a Hessian-based detector are used to detect salient features. Both types of salient features are evaluated through object categorization experiments, where only features with maximal response are retained. For most saliency criteria tested, features based on the determinant of the Hessian achieved higher classification accuracy than Harris-based features.

An Evaluation of Local Shape Descriptors in Probabilistic Volumetric Scenes

“…Recently, Knopp et al [16] have shown promising results for combined segmentation and recognition. A related study on the PVM is the work in [29], where the authors propose local descriptors based on the PCA analysis and Taylor series expansion of the appearance of local neighborhoods in the PVM. These descriptors are used in Bag of Words Models for object categorization tasks and achieve adequate classification rates when using appearance information.…”

“…In this realm, probabilistic volumetric methods offer a dense representation for the solution of the multi-view stereo problem, modeling explicitly the scene's uncertainty. This new representation of surface geometry [5,26] has been used in the areas of 2-d change detection [24,26], and more recently it has shown promising results in the areas of 3-d object classification [29]. Then, surface normals and shape descriptors are computed at highly likely surface locations.…”

“…For evaluation, this paper proposes a framework that starts by learning the probabilistic models for 17 urban scenes from publicly available aerial imagery [29]. The probabilistic models are learned using an octree-GPU-based implementation [21].…”

“…Finally, the descriptors are used to learn Bag of Words models for five object categories: planes, cars, houses, buildings and parking lots. During learning and categorization, the objects are manually segmented and labeled using the bounding boxes provided in [29]. The Bag of Words framework follows common practices, where a common vocabulary is learned for all categories using k-means clustering.…”

Shape Retrieval Through Mahalanobis Distance With Shortest Augmenting Path Algorithm

“…The result gives efficient algorithm for the shape matching with the online shape retrieval and classification. Restrepo et al (2012) presented a new volumetric representation for categorizing objects in large-scale 3-D scenes reconstructed from image sequences. This algorithm presents the first work to characterize and use the local 3-D information in the scenes.…”