A Multi-scale CNN for Affordance Segmentation in RGB Images

Roy, Anirban; Todorovic, Sinisa

doi:10.1007/978-3-319-46493-0_12

Cited by 127 publications

(78 citation statements)

References 37 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…23. http://www.semantic3d.net/ [43] GoogLeNet(FCN) Patchwise CNN, Standalone CRF CRFasRNN [70] FCN-8s CRF reformulated as RNN Dilation [71] VGG-16 Dilated convolutions ENet [72] ENet bottleneck Bottleneck module for efficiency Multi-scale-CNN-Raj [73] VGG-16(FCN) Multi-scale architecture Multi-scale-CNN-Eigen [74] Custom Multi-scale sequential refinement Multi-scale-CNN-Roy [75] Multi-scale-CNN-Eigen Multi-scale coarse-to-fine refinement Multi-scale-CNN-Bian [76] FCN Independently trained multi-scale FCNs ParseNet [77] VGG-16 Global context feature fusion ReSeg [78] VGG-16 + ReNet Extension of ReNet to semantic segmentation LSTM-CF [79] Fast R-CNN + DeepMask Fusion of contextual information from multiple sources 2D-LSTM [80] MDRNN Image context modelling rCNN [81] MDRNN Different input sizes, image context DAG-RNN [82] Elman network Graph image structure for context modelling SDS [10] R-CNN + Box CNN Simultaneous detection and segmentation DeepMask [83] VGG-A Proposals generation for segmentation SharpMask [84] DeepMask Top-down refinement module MultiPathNet [85] Fast R-CNN + DeepMask Multi path information flow through network Huang-3DCNN [86] Own 3DCNN 3DCNN for voxelized point clouds PointNet [87] Own MLP-based Segmentation of unordered point sets Clockwork Convnet [88] FCN Clockwork scheduling for sequences 3DCNN-Zhang…”

Section: Methodsmentioning

confidence: 99%

A survey on deep learning techniques for image and video semantic segmentation

García-García

Orts-Escolano

Oprea

et al. 2018

Applied Soft Computing

1,508

796

View full text Add to dashboard Cite

Image semantic segmentation is more and more being of interest for computer vision and machine learning researchers. Many applications on the rise need accurate and efficient segmentation mechanisms: autonomous driving, indoor navigation, and even virtual or augmented reality systems to name a few. This demand coincides with the rise of deep learning approaches in almost every field or application target related to computer vision, including semantic segmentation or scene understanding. This paper provides a review on deep learning methods for semantic segmentation applied to various application areas. Firstly, we describe the terminology of this field as well as mandatory background concepts. Next, the main datasets and challenges are exposed to help researchers decide which are the ones that best suit their needs and their targets. Then, existing methods are reviewed, highlighting their contributions and their significance in the field. Finally, quantitative results are given for the described methods and the datasets in which they were evaluated, following up with a discussion of the results. At last, we point out a set of promising future works and draw our own conclusions about the state of the art of semantic segmentation using deep learning techniques.

show abstract

Section: Methodsmentioning

confidence: 99%

A survey on deep learning techniques for image and video semantic segmentation

García-García

Orts-Escolano

Oprea

et al. 2018

Applied Soft Computing

1,508

796

View full text Add to dashboard Cite

show abstract

“…Scale variation typically refers to large variations in scale of the objects being counted (in this case heads) (i) within image and (ii) across images in a dataset. Several other related tasks like object detection [6,16,23,30,37,45] and visual saliency detection [10,14,41,73] are also affected by such effects. However, these effects are more evident especially in crowd counting in congested scenes.…”

Section: Introductionmentioning

confidence: 99%

Multi-Level Bottom-Top and Top-Bottom Feature Fusion for Crowd Counting

Sindagi

Patel

2019

2019 IEEE/CVF International Conference on Computer Vision (ICCV)

182

View full text Add to dashboard Cite

Crowd counting presents enormous challenges in the form of large variation in scales within images and across the dataset. These issues are further exacerbated in highly congested scenes. Approaches based on straightforward fusion of multi-scale features from a deep network seem to be obvious solutions to this problem. However, these fusion approaches do not yield significant improvements in the case of crowd counting in congested scenes. This is usually due to their limited abilities in effectively combining the multi-scale features for problems like crowd counting. To overcome this, we focus on how to efficiently leverage information present in different layers of the network. Specifically, we present a network that involves: (i) a multilevel bottom-top and top-bottom fusion (MBTTBF) method to combine information from shallower to deeper layers and vice versa at multiple levels, (ii) scale complementary feature extraction blocks (SCFB) involving cross-scale residual functions to explicitly enable flow of complementary features from adjacent conv layers along the fusion paths. Furthermore, in order to increase the effectiveness of the multi-scale fusion, we employ a principled way of generating scale-aware ground-truth density maps for training. Experiments conducted on three datasets that contain highly congested scenes (ShanghaiTech, UCF CROWD 50, and UCF-QNRF) demonstrate that the proposed method is able to outperform several recent methods in all the datasets.

show abstract

“…The network input was encoded as a HHA encoding [28] of depth along with the color image. A more generic affordance prediction framework was presented in [185], which used a multi-scale CNN to provide affordance segmentations for indoor scenes (see Figure 21). Their architecture explicitly used mid-level geometric and semantic representations such as labels, surface normals and depth maps at coarse and fine levels to effectively aggregate information.…”

Section: Methods Overviewmentioning

confidence: 99%

Indoor Scene Understanding in 2.5/3D for Autonomous Agents: A Survey

2019

View full text Add to dashboard Cite

With the availability of low-cost and compact 2.5/3D visual sensing devices, computer vision community is experiencing a growing interest in visual scene understanding of indoor environments. This survey paper provides a comprehensive background to this research topic. We begin with a historical perspective, followed by popular 3D data representations and a comparative analysis of available datasets. Before delving into the application specific details, this survey provides a succinct introduction to the core technologies that are the underlying methods extensively used in the literature. Afterwards, we review the developed techniques according to a taxonomy based on the scene understanding tasks. This covers holistic indoor scene understanding as well as subtasks such as scene classification, object detection, pose estimation, semantic segmentation, 3D reconstruction, saliency detection, physics-based reasoning and affordance prediction. Later on, we summarize the performance metrics used for evaluation in different tasks and a quantitative comparison among the recent state-of-the-art techniques. We conclude this review with the current challenges and an outlook towards the open research problems requiring further investigation.

show abstract

A Multi-scale CNN for Affordance Segmentation in RGB Images

Cited by 127 publications

References 37 publications

A survey on deep learning techniques for image and video semantic segmentation

A survey on deep learning techniques for image and video semantic segmentation

Multi-Level Bottom-Top and Top-Bottom Feature Fusion for Crowd Counting

Indoor Scene Understanding in 2.5/3D for Autonomous Agents: A Survey

Contact Info

Product

Resources

About