Automatically discovering local visual material attributes

Abstract

Shape cues play an important role in computer vision, but shape is not the only information available in images. Materials, such as fabric and plastic, are discernible in images even when shapes, such as those of an object, are not. We argue that it would be ideal to recognize materials without relying on object cues such as shape. This would allow us to use materials as a context for other vision tasks, such as object recognition. Humans are intuitively able to find visual cues that describe materials. Previous frameworks attempt to recognize these cues (as visual material traits) using fully-supervised learning. This requirement is not feasible when multiple annotators and large quantities of images are involved. In this paper, we derive a framework that allows us to discover locally-recognizable material attributes from crowdsourced perceptual material distances. We show that the attributes we discover do in fact separate material categories. Our learned attributes exhibit the same desirable properties as material traits, despite the fact that they are discovered using only partial supervision.

From BoW to CNN: Two Decades of Texture Representation for Texture Classification

“…For a large number of applications an ability to analyze small image sizes at high speed is vital, including facial image analysis, interest region description, segmentation, defect detection, and tracking. Many existing texture descriptors would fail in this respect, and it would be important to evaluate the performance of new descriptors [205]. Table 4 Performance (%) summarization of some representative methods on popular benchmark texture datasets.…”

Differential Angular Imaging for Material Recognition

“…This method relies on large image patches that include object and scene context to recognize materials. In contrast, Schwartz and Nishino [34,35] learn material appearance models from small image patches extracted inside object boundaries to decouple contextual information from material appearance. To achieve accurate local material recognition, they introduced intermediate material appearance representations based on their intrinsic properties (e.g., "smooth" and "metallic").…”

Integrating deep features for material recognition

“…The authors achieved stateof-the-art performance on several benchmark datasets for texture recognition and material recognition, including 82.4% accuracy on the FMD dataset. In [8], a method was proposed to discover local material attributes from crowdsourced perceptual material distances. They show that without relying on object cues (e.g., outlines, shapes), material recognition can still be performed by employing the discovered local material attributes.…”

“…and set equal weight w i = 1 to each sample in C. -For each individual feature of x c , define the sets F = {x c,j ∈ x c } |xc| j=1 , and S = ∅. 1 for j ← 1 to |F| do 2 Construct K samples that have maximal feature values on x c,j using (1); 3 Compute class entropy on the set of top K samples using (2);Select the feature that minimizes the weighted class entropy using (3);8 Penalize the samples using (4); Integration of features extracted using deep representations of CNNs.…”