Fine-Grained Categorization for 3D Scene Understanding

Abstract

Fine-grained categorization of object classes is receiving increased attention, since it promises to automate classification tasks that are difficult even for humans, such as the distinction between different animal species. In this paper, we consider fine-grained categorization for a different reason: following the intuition that fine-grained categories encode metric information, we aim to generate metric constraints from fine-grained category predictions, for the benefit of 3D scene-understanding. To that end, we propose two novel methods for fine-grained classification, both based on part information, as well as a new fine-grained category data set of car types. We demonstrate superior performance of our methods to state-of-the-art classifiers, and show first promising results for estimating the depth of objects from fine-grained category predictions from a monocular camera.

Jointly Optimizing 3D Model Fitting and Fine-Grained Classification

“…Existing fine-grained car datasets (e.g., [13,23]) are not suitable for our purposes, since they are not annotated with both landmark locations and fine-grained class labels. We created a new fine-grained 3D car dataset (FG3DCar) for this study 3 , which consists of 300 images with 30 different car models under different viewing angles, e.g., sedan, SUV, crossover, hatchback, wagon and pickup truck.…”

“…To validate the effectiveness of these baseline methods, we apply them on a public fine-grained car dataset [23] (denoted as BMVC dataset). Experimental results (left column in Table. 1) show that they achieve very competitive results Table 1.…”

“…4 is learnt by averaging the shape parameters within the same class from our training dataset. Foreground images are used, following standard criteria in fine-grained classification [23,14], and resized to height = 300 pixels. Note that the reported numbers in the following experiments (e.g., fitting accuracy) are based on this image scale.…”

“…Fine-grained recognition methods have been proposed to address different types of super-ordinate categories (e.g., birds [10,5,6,8], dogs [18] or cars [23,14]), and many of these methods focus on finding distinctive 2D parts for distinguishing different classes [6,27,5,2] or seeking better pose-invariant feature representations [29]. Recently, researchers [8,14] have used 3D models for fine-grained classification.…”

Predicting an Object Location Using a Global Image Representation

“…In mobile phone applications, such as product search [28] or leafsnap.com [16], users take pictures of an "object of interest", and localizing the object is often necessary for the subsequent processing steps. Another scenario where this problem is found is fine-grained categorization [8,29,36,35]. In fine-grained categorization an image contains an object of a parent class (e.g.…”