The Role of Context for Object Detection and Semantic Segmentation in the Wild

Mottaghi, Roozbeh; Chen, Xianjie; Liu, Xiaobai; Cho, Nam Gyu; Lee, Seong–Whan; Fidler, Sanja; Urtasun, Raquel; Yuille, Alan

doi:10.1109/cvpr.2014.119

Cited by 1,249 publications

(931 citation statements)

References 22 publications

Supporting

Mentioning

927

Contrasting

Unclassified

Order By: Relevance

“…For instance, surrounding background usually offers strong cues for object recognition, sky and ground usually appear at predictable locations in a scene, and objects are made up of known parts at familiar relative locations. Such structural information within visual data has been used to improve inference in several problems, such as object detection and semantic segmentation [31,35,29].…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

DeepPermNet: Visual Permutation Learning

Cruz

Fernando

Cherian

et al. 2017

2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)

View full text Add to dashboard Cite

We present a principled approach to uncover the structure of visual data by solving a novel deep learning task coined visual permutation learning. The goal of this task is to find the permutation that recovers the structure of data from shuffled versions of it. In the case of natural images, this task boils down to recovering the original image from patches shuffled by an unknown permutation matrix. Unfortunately, permutation matrices are discrete, thereby posing difficulties for gradient-based methods. To this end, we resort to a continuous approximation of these matrices using doubly-stochastic matrices which we generate from standard CNN predictions using Sinkhorn iterations. Unrolling these iterations in a Sinkhorn network layer, we propose DeepPermNet, an end-to-end CNN model for this task.The utility of DeepPermNet is demonstrated on two challenging computer vision problems, namely, (i) relative attributes learning and (ii) self-supervised representation learning. Our results show state-of-the-art performance on the Public Figures and OSR benchmarks for (i) and on the classification and segmentation tasks on the PASCAL VOC dataset for (ii).

show abstract

Section: Introductionmentioning

confidence: 99%

“…Such an in-ference could be used to understand scene structure, visual attributes, or semantic concepts in images or image regions. The knowledge acquired by this sort of learning framework can then be used to solve many other computer vision tasks, such as learning-to-rank [15], image reconstruction [7] and object segmentation [31].…”

Section: Introductionmentioning

confidence: 99%

DeepPermNet: Visual Permutation Learning

Cruz

Fernando

Cherian

et al. 2017

2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)

View full text Add to dashboard Cite

show abstract

“…These measures are evaluated on two types of natural image datasets: 1) Pascal Context segmentation dataset [31], and 2) MS-COCO [32] and Pascal VOC'12 [33] object segmentation datasets. Compared to the classical approach for hierarchy evaluation that focuses only on the horizontal cuts and the image segmentation problem, we believe that the proposed framework offers a richer assessment that better accounts for the hierarchical nature of the representations and it is not limited to a single use case.…”

Section: Introductionmentioning

confidence: 99%

Evaluation of Hierarchical Watersheds

Perret¹,

Cousty²,

Guimarães³

et al. 2018

IEEE Trans. on Image Process.

View full text Add to dashboard Cite

Abstract-This article aims to understand the practical features of hierarchies of morphological segmentations, namely the quasi-flat zones hierarchy and watershed hierarchies, and to evaluate their potential in the context of natural image analysis. We propose a novel evaluation framework for hierarchies of partitions designed to capture various aspects of those representations: precision of their regions and contours, possibility to extract high quality horizontal cuts and optimal non-horizontal cuts for image segmentation, and ease of finding a set of regions representing a semantic object. This framework is used to assess and to optimize hierarchies with respect to the possible pre-and post-processing steps. We show that, used in conjunction with a state-of-the-art contour detector, watershed hierarchies are competitive with complex state-of-the-art methods for hierarchy construction. In particular, the proposed framework allows us to identify a watershed hierarchy based on a novel extinction value, the number of parent nodes, that outperforms the other hierarchies of morphological segmentations. This coupled with the fact that watershed hierarchies satisfy clear global optimality properties and can be efficiently computed on large data, make them valuable candidates for various computer vision tasks.Index Terms-mathematical morphology, hierarchy of partitions, watershed segmentation, image analysis.

show abstract

“…For instance, given an image of a sheep and a dog, the attention mechanism is that which determines what the desired output is when a specific image location is attended to. This is further complicated by the role of context in recognition, as the context in which objects appear can have a significant effect on how they are perceived (see for instance (Mottaghi et al, 2014)). …”

Section: Attention Models In Deep Networkmentioning

confidence: 99%

“…An improved target segmentation based on (Mishra and Aloimonos, 2009) was added for experiments on attention masking in deep networks. The classifier used on the platform for these experiments was a pre-trained deep neural network with frozen convolutional layers from the VGG-F network (Chatfield et al, 2014), with an added spatial pyramid pooling (SPP) layer (He et al, 2014) and fully-connected layers trained on the PASCAL-Context dataset (Mottaghi et al, 2014). The details of these experiments can be found in (Wallenberg and Forssén, 2017).…”

Section: Later Experimentsmentioning

confidence: 99%

Embodied Visual Object Recognition

Wallenberg¹

View full text Add to dashboard Cite

Object recognition is a skill we as humans often take for granted. Due to our formidable object learning, recognition and generalisation skills, it is sometimes hard to see the multitude of obstacles that need to be overcome in order to replicate this skill in an artificial system. Object recognition is also one of the classical areas of computer vision, and many ways of approaching the problem have been proposed.Recently, visually capable robots and autonomous vehicles have increased the focus on embodied recognition systems and active visual search. These applications demand that systems can learn and adapt to their surroundings and arrive at decisions in a reasonable amount of time. Ideally, this should be done while maintaining high object recognition performance. This is especially challenging due to the high dimensionality of image data and in cases where end-to-end learning from pixels to output is needed. Therefore, mechanisms designed to make inputs tractable are often necessary for less computationally capable embodied systems.Active visual search also means that mechanisms for attention and gaze control are integral to the object recognition procedure. Therefore, the way in which attention mechanisms should be introduced into feature extraction and estimation algorithms must be carefully considered when constructing a recognition system. This thesis describes work done on the components necessary for creating an embodied recognition system, specifically in the areas of decision uncertainty estimation, object segmentation from multiple cues, adaptation of stereo vision to a specific platform and setting, problem-specific feature selection, efficient estimator training and attentional modulation in convolutional neural networks. Contributions include the evaluation of methods and measures for predicting the potential uncertainty reduction that can be obtained from additional views of an object, allowing for adaptive target observations. Also, in order to separate a specific object from other parts of a scene, it is often necessary to combine multiple cues such as colour and depth in order to obtain satisfactory results. Therefore, a method for combining these using channel coding has been evaluated. In order to make use of three-dimensional spatial structure in recognition, a novel stereo vision algorithm extension along with a framework for automatic stereo tuning have also been investigated. Feature selection and efficient discriminant sampling for decision tree-based estimators have also been implemented. Finally, attentional multi-layer modulation of convolutional neural networks for recognition in cluttered scenes has been investigated. Several of these components have been tested and evaluated on a purpose-built embodied recognition platform known as Eddie the Embodied.v Populärvetenskaplig sammanfattningFörmågan att tolka och lära av synintryck är både viktig och självklar för oss män-niskor. Detta gör att vi sällan ägnar en tanke åt den enorma komplexitet och de många utmaningar vi med l...

show abstract

The Role of Context for Object Detection and Semantic Segmentation in the Wild

Cited by 1,249 publications

References 22 publications

DeepPermNet: Visual Permutation Learning

DeepPermNet: Visual Permutation Learning

Evaluation of Hierarchical Watersheds

Embodied Visual Object Recognition

Contact Info

Product

Resources

About