DEEP-CARVING: Discovering visual attributes by carving deep neural nets

Shankar, Sukrit; Garg, Vikas; Cipolla, Roberto

doi:10.1109/cvpr.2015.7298962

Cited by 45 publications

(37 citation statements)

References 44 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Recently, Shankar et al [23] proposed a modified CNN training procedure to improve attribute recognition. Their "deep carving" algorithm provides the CNN with attribute pseudolabel targets, updated periodically during training.…”

Section: Materials Perception and Convolutional Neural Networkmentioning

confidence: 99%

“…Similarly for conventional object and scene recognition, attributes like "sunset" or "natural," have also been extracted for use as independent features. Shankar et al [23] generate pseudo-labels to improve the attribute prediction accuracy of a Convolutional Neural Network, and Zhou et al [25] discover concepts from weakly-supervised image data. In both cases, the attributes are considered on their own, not within the context of higher-level categories.…”

Section: Perceptual Materials Attributes In Con-volutional Neural Netwmentioning

confidence: 99%

See 1 more Smart Citation

Recognizing Material Properties from Images

Schwartz

Nishino

2020

IEEE Trans. Pattern Anal. Mach. Intell.

View full text Add to dashboard Cite

Humans implicitly rely on properties of the materials that make up ordinary objects to guide our interactions. Grasping smooth materials, for example, requires more care than rough ones, and softness is an ideal property for fabric used in bedding. Even when these properties are not purely visual (softness is a physical property of the material), we may still infer the softness of a fabric by looking at it. We refer to these visually-recognizable material properties as visual material attributes. Recognizing visual material attributes in images can contribute valuable information for general scene understanding and for recognition of materials themselves. Unlike wellknown object and scene attributes, visual material attributes are local properties. "Fuzziness", for example, does not have a particular shape. We show that given a set of images annotated with known material attributes, we may accurately recognize the attributes from purely local information (small image patches). Obtaining such annotations in a consistent fashion at scale, however, is challenging. We introduce a method that allows us to solve this problem by probing the human visual perception of materials to automatically discover unnamed attributes that serve the same purpose. By asking simple yes/no questions comparing pairs of image patches, we obtain sufficient weak supervision to build a set of attributes (and associated classifiers) that, while being unnamed, serve the same function as the named attributes, such as "fuzzy" or "rough", with which we describe materials. Doing so allows us to recognize visual material attributes without resorting to exhaustive manual annotation of a fixed set of named attributes. Furthermore, we show that our automatic attribute discovery method may be integrated in the end-to-end learning of a material classification CNN framework to simultaneously recognize materials and discover their visual material attributes. Our experimental results show that visual material attributes, whether named or automatically discovered, provide a useful intermediate representation for known material categories themselves as well as a basis for transfer learning when recognizing previously-unseen categories.Index Terms-visual material attributes, human material perception, material recognition ! The authors are with the

show abstract

Section: Materials Perception and Convolutional Neural Networkmentioning

confidence: 99%

Section: Perceptual Materials Attributes In Con-volutional Neural Netwmentioning

confidence: 99%

Recognizing Material Properties from Images

Schwartz

Nishino

2020

IEEE Trans. Pattern Anal. Mach. Intell.

View full text Add to dashboard Cite

show abstract

“…Earlier work relied on hand-crafted features like SIFT and HOG to model the attributes [2,19,20,1,21,3,22,23,6]. More recent work use deep convolutional networks to learn the attribute representations, and achieve superior performance [24,25,9,26]. While these approaches learn deep representations for binary attributes, we instead learn deep representations for relative attributes.…”

Section: Related Workmentioning

confidence: 99%

End-to-End Localization and Ranking for Relative Attributes

Singh

Lee

2016

Computer Vision – ECCV 2016

View full text Add to dashboard Cite

Abstract. We propose an end-to-end deep convolutional network to simultaneously localize and rank relative visual attributes, given only weakly-supervised pairwise image comparisons. Unlike previous methods, our network jointly learns the attribute's features, localization, and ranker. The localization module of our network discovers the most informative image region for the attribute, which is then used by the ranking module to learn a ranking model of the attribute. Our end-to-end framework also significantly speeds up processing and is much faster than previous methods. We show state-of-the-art ranking results on various relative attribute datasets, and our qualitative localization results clearly demonstrate our network's ability to learn meaningful image patches.

show abstract

“…The computer vision community has explored scene understanding from the perspective of scene classification [20,39], attribute prediction [24,32,39], geometry prediction [15] and pixel level semantic segmentation [7,25,38]. All these approaches, however, only reason about information directly present in the scene.…”

Section: Related Workmentioning

confidence: 99%

DeepNav: Learning to Navigate Large Cities

Brahmbhatt

Hays

2017

2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)

View full text Add to dashboard Cite

We present DeepNav, a Convolutional Neural Network (CNN) based algorithm for navigating large cities using locally visible street-view images. The DeepNav agent learns to reach its destination quickly by making the correct navigation decisions at intersections. We collect a large-scale dataset of street-view images organized in a graph where nodes are connected by roads. This dataset contains 10 city graphs and more than 1 million street-view images. We propose 3 supervised learning approaches for the navigation task and show how A* search in the city graph can be used to generate supervision for the learning. Our annotation process is fully automated using publicly available mapping services and requires no human input. We evaluate the proposed DeepNav models on 4 held-out cities for navigating to 5 different types of destinations. Our algorithms outperform previous work that uses hand-crafted features and Support Vector Regression (SVR) [19].

show abstract

DEEP-CARVING: Discovering visual attributes by carving deep neural nets

Cited by 45 publications

References 44 publications

Recognizing Material Properties from Images

Recognizing Material Properties from Images

End-to-End Localization and Ranking for Relative Attributes

DeepNav: Learning to Navigate Large Cities

Contact Info

Product

Resources

About