ReSeg: A Recurrent Neural Network-Based Model for Semantic Segmentation

Visin, Francesco; Romero, Adriana; Cho, Kyunghyun; Matteucci, Matteo; Ciccone, Marco Matteo; Kastner, Kyle; Bengio, Yoshua; Courville, Aaron

doi:10.1109/cvprw.2016.60

Cited by 239 publications

(159 citation statements)

References 39 publications

Supporting

Mentioning

158

Contrasting

Unclassified

Order By: Relevance

“…Extending this to 2 dimensions would make RCNs more suitable for processing images, and 3 dimensions could enable fast learning for video processing. Combining bi-directional (horizontal and vertical) scanning as done in this paper and in [36] does provide some of the benefits of 2D convolution, but we expect that such setup will prove to be sub-optimal for larger image sizes. Extensions that better mimic the local properties of 2D convolution are however non-trivial and, to our best knowledge, no such extensions have been proposed yet.…”

Section: Conclusion Discussion and Future Workmentioning

confidence: 99%

On the application of reservoir computing networks for noisy image recognition

et al. 2018

View full text Add to dashboard Cite

Reservoir Computing Networks (RCNs) are a special type of single layer recurrent neural networks, in which the input and the recurrent connections are randomly generated and only the output weights are trained. Besides the ability to process temporal information, the key points of RCN are easy training and robustness against noise. Recently, we introduced a simple strategy to tune the parameters of RCNs. Evaluation in the domain of noise robust speech recognition proved that this method was effective. The aim of this work is to extend that study to the field of image processing, by showing that the proposed parameter tuning procedure is equally valid in the field of image processing and conforming that RCNs are apt at temporal modeling and are robust with respect to noise. In particular, we investigate the potential of RCNs in achieving competitive performance on the well-known MNIST dataset by following the aforementioned parameter optimizing strategy. Moreover, we achieve good noise robust recognition by utilizing such a network to denoise images and supplying them to a recognizer that is solely trained on clean images. The experiments demonstrate that the proposed RCN-based handwritten digit recognizer achieves an error rate of 0.81 percent on the clean test data of the MNIST benchmark and that the proposed RCN-based denoiser can effectively reduce the error rate on the various types of noise.

show abstract

Section: Conclusion Discussion and Future Workmentioning

confidence: 99%

On the application of reservoir computing networks for noisy image recognition

et al. 2018

View full text Add to dashboard Cite

show abstract

“…RNN models have also proven to be effective for tasks with densely connected data such as semantic segmentation [76], scene parsing [51] and even as an alternative to Convolutional Neural Networks [65]. These works show that RNN models are capable of learning the dependencies between spatially correlated data such as image pixels.…”

Section: Related Workmentioning

confidence: 99%

Social LSTM: Human Trajectory Prediction in Crowded Spaces

Alahi

Goel

Ramanathan

et al. 2016

2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)

2,736

2,594

View full text Add to dashboard Cite

Pedestrians follow different trajectories to avoid obstacles and accommodate fellow pedestrians. Any autonomous vehicle navigating such a scene should be able to foresee the future positions of pedestrians and accordingly adjust its path to avoid collisions. This problem of trajectory prediction can be viewed as a sequence generation task, where we are interested in predicting the future trajectory of people based on their past positions. Following the recent success of Recurrent Neural Network (RNN) models for sequence prediction tasks, we propose an LSTM model which can learn general human movement and predict their future trajectories. This is in contrast to traditional approaches which use hand-crafted functions such as Social forces. We demonstrate the performance of our method on several public datasets. Our model outperforms state-of-the-art methods on some of these datasets . We also analyze the trajectories predicted by our model to demonstrate the motion behaviour learned by our model.

show abstract

“…For example, Zuo et al [54] converted each image into 1D spatial sequences by concatenating the CNN features of different regions, and utilized RNN to learn the spatial dependencies of image regions. Similar work appeared in [46]. The proposed ReNet replaced the ubiquitous convolutional+pooling layer with four recurrent neural networks that sweep horizontally and vertically in both directions across the image.…”

Section: Usage Of Cnn-rnn Frameworkmentioning

confidence: 85%

CNN-RNN: a large-scale hierarchical image classification framework

Guo¹,

Liu²,

Bakker³

et al. 2017

Multimed Tools Appl

122

View full text Add to dashboard Cite

Objects are often organized in a semantic hierarchy of categories, where finelevel categories are grouped into coarse-level categories according to their semantic relations. While previous works usually only classify objects into the leaf categories, we argue that generating hierarchical labels can actually describe how the leaf categories evolved from higher level coarse-grained categories, thus can provide a better understanding of the objects. In this paper, we propose to utilize the CNN-RNN framework to address the hierarchical image classification task. CNN allows us to obtain discriminative features for the input images, and RNN enables us to jointly optimize the classification of coarse and fine labels. This framework can not only generate hierarchical labels for images, but also improve the traditional leaf-level classification performance due to incorporating the hierarchical information. Moreover, this framework can be built on top of any CNN architecture which is primarily designed for leaf-level classification. Accordingly, we build a high performance network based on the CNN-RNN paradigm which outperforms the original CNN (wider-ResNet) and also the current state-of-the-art. In addition, we investigate how to utilize the CNN-RNN framework to improve the fine category classification when a fraction of the training data is only annotated with coarse labels. Experimental results demonstrate that CNN-RNN can use the coarse-labeled training data to improve the classification of fine categories, and in some cases it even surpasses the performance achieved by fully annotated training data. This reveals that, CNN-RNN can alleviate the challenge of specialized and expensive annotation of fine labels.

show abstract

ReSeg: A Recurrent Neural Network-Based Model for Semantic Segmentation

Cited by 239 publications

References 39 publications

On the application of reservoir computing networks for noisy image recognition

On the application of reservoir computing networks for noisy image recognition

Social LSTM: Human Trajectory Prediction in Crowded Spaces

CNN-RNN: a large-scale hierarchical image classification framework

Contact Info

Product

Resources

About