Stacked Convolutional Auto-Encoders for Hierarchical Feature Extraction

Masci, Jonathan; Meier, Ueli; Cireşan, Dan; Schmidhuber, Juergen

doi:10.1007/978-3-642-21735-7_7

Cited by 1,537 publications

(1,082 citation statements)

References 21 publications

Supporting

Mentioning

1,066

Contrasting

Unclassified

Order By: Relevance

“…The hidden layer, as extracted features, were used as input data for the next encoding to acquire features in the next layer and, subsequently, the decoder was used to reconstruct the images layer by layer, starting from the bottom hidden layer. Based on SAE, SCAE changed all the input, output and the hidden layers with a one-dimension structure and used the convolution network for improved conservation of the spatial features [12]. Similar to the traditional CNN network, SCAE is the stacking of several building blocks [13], with each block containing a convolutional layer, pooling layer and a nonlinearity layer.…”

Section: Feature Extractionmentioning

confidence: 99%

GAN-Assisted Two-Stream Neural Network for High-Resolution Remote Sensing Image Classification

Tao

Zhong

et al. 2017

Remote Sensing

View full text Add to dashboard Cite

Abstract:Using deep learning to improve the capabilities of high-resolution satellite images has emerged recently as an important topic in automatic classification. Deep networks track hierarchical high-level features to identify objects; however, enhancing the classification accuracy from low-level features is often disregarded. We therefore proposed a two-stream deep-learning neural network strategy, with a main stream utilizing fine spatial-resolution panchromatic images to retain low-level information under a supervised residual network structure. An auxiliary line employed an unsupervised net to extract high-level abstract and discriminative features from multispectral images to supplement the spectral information in the main stream. Various feature extraction types from the neural network were selected and jointed in the novel net, as the combined high-and low-level features could provide a superior solution to image classification. In traditional convolutional neural networks, increased network depth might not influence the network performance perceptibly; however, we introduced a residual neural network to develop the expressive ability of the deeper net, increasing the role of net depth in feature extraction. To enhance feature robustness, we proposed a novel consolidation part in feature extraction. An adversarial net improved the feature extraction capabilities and aided digging the inherent and discriminative features from data, with increased extraction efficacy. Tests on satellite images indicated the high overall accuracy of our novel net, verifying that net depth or number of convolution kernels affected the classification capability. Various comparative tests proved the structural rationality for our two-stream structure.

show abstract

Section: Feature Extractionmentioning

confidence: 99%

GAN-Assisted Two-Stream Neural Network for High-Resolution Remote Sensing Image Classification

Tao

Zhong

et al. 2017

Remote Sensing

View full text Add to dashboard Cite

show abstract

“…Several approaches involving the combination of these methods have been explored in the past, and here we use a CAE architecture along the lines presented in [11,18,17].…”

Section: Convolutional Autoencodersmentioning

confidence: 99%

DeepPainter: Painter Classification Using Deep Convolutional Autoencoders

David

Netanyahu

2016

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Abstract. In this paper we describe the problem of painter classification, and propose a novel approach based on deep convolutional autoencoder neural networks. While previous approaches relied on image processing and manual feature extraction from paintings, our approach operates on the raw pixel level, without any preprocessing or manual feature extraction. We first train a deep convolutional autoencoder on a dataset of paintings, and subsequently use it to initialize a supervised convolutional neural network for the classification phase. The proposed approach substantially outperforms previous methods, improving the previous state-of-the-art for the 3-painter classification problem from 90.44% accuracy (previous state-of-the-art) to 96.52% accuracy, i.e., a 63% reduction in error rate.

show abstract

“…The closest that unsupervised pre-training has come to FCN architectures, to the best of our knowledge, is stacked convolutional autoencoders, as defined by Mesci et al [2]. A convolutional autoencoder is a convolutional layer that is required to reconstruct its input after applying a pooling operation over its feature maps (to discourage the trivial solution), and are typically trained using the standard greedy layer-wise approach.…”

Section: Related Workmentioning

confidence: 99%

“…These transformations were possible horizontal mirroring, rotations by multiples of 10 • , and elastic deformations using parameters sampled from a continuous distribution 2 . This sampling ensures that any specific transformed image is extremely unlikely to reoccur during training, thus significantly reducing the risk of overfitting.…”

Section: Data Setmentioning

confidence: 99%

See 1 more Smart Citation

Unsupervised pre-training for fully convolutional neural networks

Wiehman

Kroon

Villiers

2016

2016 Pattern Recognition Association of South Africa and Robotics and Mechatronics International Conference (PRASA-RobMech)

View full text Add to dashboard Cite

Abstract-Unsupervised pre-training of neural networks has been shown to act as a regularization technique, improving performance and reducing model variance. Recently, fully convolutional networks (FCNs) have shown state-of-the-art results on various semantic segmentation tasks. Unfortunately, there is no efficient approach available for FCNs to benefit from unsupervised pre-training. Given the unique property of FCNs to output segmentation maps, we explore a novel variation of unsupervised pre-training specifically designed for FCNs. We extend an existing FCN, called U-net, to facilitate end-to-end unsupervised pre-training and apply it on the ISBI 2012 EM segmentation challenge data set. We performed a battery of significance tests for both equality of means and equality of variance, and show that our results are consistent with previous work on unsupervised pre-training obtained from much smaller networks. We conclude that end-to-end unsupervised pre-training for FCNs adds robustness to random initialization, thus reducing model variance.

show abstract

Stacked Convolutional Auto-Encoders for Hierarchical Feature Extraction

Cited by 1,537 publications

References 21 publications

GAN-Assisted Two-Stream Neural Network for High-Resolution Remote Sensing Image Classification

GAN-Assisted Two-Stream Neural Network for High-Resolution Remote Sensing Image Classification

DeepPainter: Painter Classification Using Deep Convolutional Autoencoders

Unsupervised pre-training for fully convolutional neural networks

Contact Info

Product

Resources

About