Indoor Scene Classification Using Combined 3D and Gist Features

Abstract

Abstract. Scene categorization is an important mechanism for providing high-level context which can guide methods for a more detailed analysis of scenes. State-of-the-art techniques like Torralba's Gist features show a good performance on categorizing outdoor scenes but have problems in categorizing indoor scenes. In contrast to object based approaches, we propose a 3D feature vector capturing general properties of the spatial layout of indoor scenes like shape and size of extracted planar patches and their orientation to each other. This idea is supported by psychological experiments which give evidence for the special role of 3D geometry in categorizing indoor scenes. In order to study the influence of the 3D geometry we introduce in this paper a novel 3D indoor database and a method for defining 3D features on planar surfaces extracted in 3D data. Additionally, we propose a voting technique to fuse 3D features and 2D Gist features and show in our experiments a significant contribution of the 3D features to the indoor scene categorization task.

A comparative study of rice variety classification based on deep learning and hand-crafted features

“…Therefore, 32 × 16 = 512 GIST features are extracted from each image. GIST has been successfully applied for indoor/outdoor scene recognition [60] [61], traffic scene classification [62], and face recognition [63][64].…”

Data-driven shape analysis and processing

“…Inferring scene semantics is a long-standing problem in image understanding, with many methods developed for object recognition [QT09], classification [SW10], inferring spatial layout [CCPS13], and other 3D information [FGH13] from a single image. Previous work demonstrates that one can leverage collections of 3D models to facilitate scene understanding in images [SLH12].…”

Leveraging Human Computations to Improve Schematization of Spatial Relations from Imagery

“…The general procedure for automatically reconstructing a 3D scene typically consists of two steps: object detection and depth estimation. Visual features are extracted using various techniques in the area of computer vision to detect salient objects from a picture of a scene [Bay et al 2006;Espinace et al 2010;Swadzba and Wachsmuth 2011]. Typical techniques to extract distance information (e.g., depth information)…”