Attention to Scale: Scale-aware Semantic Image Segmentation

Chen, Liang-Chieh; Yang, Yi; Wang, Jiang; Xu, Wei; Yuille, Alan

doi:10.48550/arxiv.1511.03339

Cited by 40 publications

(40 citation statements)

References 3 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Evaluation metric: The standard intersection over union (IOU) criterion and pixelwise accuracy are adopted for evaluation on PASCAL-Person-Part dataset and Horse-Cow parsing dataset, following [21][31] [9]. We use the same evaluation metrics as in [4] [22][36] for evaluation on two human parsing datasets, including accuracy, average precision, average recall, and average F-1 score.…”

Section: Methodsmentioning

confidence: 99%

“…Compared to Tree-LSTM, Graph LSTM is more natural and general for 2D image processing with arbitrary graph topologies and adaptive updating schemes. Semantic Object Parsing: There has been increasing research interest on the semantic object parsing problem including the general object parsing [21][12] [30][20] [31], person part segmentation [9][8] and human parsing [23][4] [32][33][34] [35] [36]. To capture the rich structure information based on the advanced CNN architecture, one common way is the combination of CNNs and CRFs [11][6][37] [12], where the CNN outputs are treated as unary potentials while CRF further incorporates pairwise or higher order factors.…”

Section: Related Workmentioning

confidence: 99%

“…Recently, Convolutional Neural Networks (CNNs) have demonstrated exciting success in various pixel-wise prediction tasks such as semantic segmentation [6] [7], semantic part segmentation [8] [9] and depth prediction [10]. However, the pure convolutional filters can only capture limited local context while the precise inference for semantic part layouts and their interactions requires a global perspective of the image.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Semantic Object Parsing with Graph LSTM

Liang¹,

Shen

Feng

et al. 2016

Lecture Notes in Computer Science

314

197

View full text Add to dashboard Cite

By taking the semantic object parsing task as an exemplar application scenario, we propose the Graph Long Short-Term Memory (Graph LSTM) network, which is the generalization of LSTM from sequential data or multidimensional data to general graph-structured data. Particularly, instead of evenly and fixedly dividing an image to pixels or patches in existing multi-dimensional LSTM structures (e.g., Row, Grid and Diagonal LSTMs [1][2]), we take each arbitrary-shaped superpixel as a semantically consistent node, and adaptively construct an undirected graph for each image, where the spatial relations of the superpixels are naturally used as edges. Constructed on such an adaptive graph topology, the Graph LSTM is more naturally aligned with the visual patterns in the image (e.g., object boundaries or appearance similarities) and provides a more economical information propagation route. Furthermore, for each optimization step over Graph LSTM, we propose to use a confidence-driven scheme to update the hidden and memory states of nodes progressively till all nodes are updated. In addition, for each node, the forgets gates are adaptively learned to capture different degrees of semantic correlation with neighboring nodes. Comprehensive evaluations on four diverse semantic object parsing datasets well demonstrate the significant superiority of our Graph LSTM over other state-of-the-art solutions.

show abstract

Section: Methodsmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Semantic Object Parsing with Graph LSTM

Liang¹,

Shen

Feng

et al. 2016

Lecture Notes in Computer Science

314

197

View full text Add to dashboard Cite

show abstract

“…Incorporating multi-scale inputs State-of-art models on the PASCAL VOC 2012 leaderboard usually employ multiscale features (either multi-scale inputs [10,28,7] or features from intermediate layers of DCNN [31,19,5]). Motivated by this, we further combine our proposed discriminatively trained domain transform and the model of [7], yielding 76.3% performance on test set, 1.5% behind current best models [28] which jointly train CRF and DCNN [6] EdgeNet on BSDS500 We further evaluate the edge detection performance of our learned EdgeNet on the test set of BSDS500 [1]. We employ the standard metrics to evaluate edge detection accuracy: fixed contour threshold (ODS Fscore), per-image best threshold (OIS F-score), and average precision (AP).…”

Section: Models Pretrained With Ms-cocomentioning

confidence: 99%

Semantic Image Segmentation with Task-Specific Edge Detection Using CNNs and a Discriminatively Trained Domain Transform

Chen

Barron

Papandreou

et al. 2016

2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)

Self Cite

323

184

View full text Add to dashboard Cite

Deep convolutional neural networks (CNNs) are the backbone of state-of-art semantic image segmentation systems. Recent work has shown that complementing CNNs with fully-connected conditional random fields (CRFs) can significantly enhance their object localization accuracy, yet dense CRF inference is computationally expensive. We propose replacing the fully-connected CRF with domain transform (DT), a modern edge-preserving filtering method in which the amount of smoothing is controlled by a reference edge map. Domain transform filtering is several times faster than dense CRF inference and we show that it yields comparable semantic segmentation results, accurately capturing object boundaries. Importantly, our formulation allows learning the reference edge map from intermediate CNN features instead of using the image gradient magnitude as in standard DT filtering. This produces task-specific edges in an end-to-end trainable system optimizing the target semantic segmentation quality.

show abstract

“…As it has been widely confirmed that feeding multiple scales of an input image to networks with shared parameters are beneficial for accurately localizing objects of different scales in pixel labeling problems [15,8,14,32], we replicate the refined VGG network in the previous section three times, each responsible for one of the scales. An input image is resized to three different scales (s ∈ {1, 0.75, 0.5}).…”

Section: Multiscale Fusion With Attentional Weightsmentioning

confidence: 99%

Instance-Level Salient Object Segmentation

Xie

et al. 2017

2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)

238

107

View full text Add to dashboard Cite

Image saliency detection has recently witnessed rapid progress due to deep convolutional neural networks. However, none of the existing methods is able to identify object instances in the detected salient regions. In this paper, we present a salient instance segmentation method that produces a saliency mask with distinct object instance labels for an input image. Our method consists of three steps, estimating saliency map, detecting salient object contours and identifying salient object instances. For the first two steps, we propose a multiscale saliency refinement network, which generates high-quality salient region masks and salient object contours. Once integrated with multiscale combinatorial grouping and a MAP-based subset optimization framework, our method can generate very promising salient object instance segmentation results. To promote further research and evaluation of salient instance segmentation, we also construct a new database of 1000 images and their pixelwise salient instance annotations. Experimental results demonstrate that our proposed method is capable of achieving state-of-the-art performance on all public benchmarks for salient region detection as well as on our new dataset for salient instance segmentation.

show abstract

Attention to Scale: Scale-aware Semantic Image Segmentation

Cited by 40 publications

References 3 publications

Semantic Object Parsing with Graph LSTM

Semantic Object Parsing with Graph LSTM

Semantic Image Segmentation with Task-Specific Edge Detection Using CNNs and a Discriminatively Trained Domain Transform

Instance-Level Salient Object Segmentation

Contact Info

Product

Resources

About