Advances in optimizing recurrent networks

Bengio, Yoshua; Boulanger-Lewandowski, Nicolas; Pascanu, Razvan

doi:10.1109/icassp.2013.6639349

Cited by 380 publications

(254 citation statements)

References 13 publications

Supporting

Mentioning

253

Contrasting

Order By: Relevance

“…To train the models we used minibatch gradient descent with a batch size 8 of 16 and Nesterov momentum (Bengio et al 2013) with coefficient µ = 0.9. Nesterov momentum is a method for accelerating gradient descent by accumulating gradients over time in directions that consistently decrease the objective function value.…”

Section: Trainingmentioning

confidence: 99%

Rotation-invariant convolutional neural networks for galaxy morphology prediction

Dieleman¹,

Willett²,

Dambre³

2015

Monthly Notices of the Royal Astronomical Society

728

604

View full text Add to dashboard Cite

Measuring the morphological parameters of galaxies is a key requirement for studying their formation and evolution. Surveys such as the Sloan Digital Sky Survey (SDSS) have resulted in the availability of very large collections of images, which have permitted population-wide analyses of galaxy morphology. Morphological analysis has traditionally been carried out mostly via visual inspection by trained experts, which is time-consuming and does not scale to large ( 10 4 ) numbers of images.Although attempts have been made to build automated classification systems, these have not been able to achieve the desired level of accuracy. The Galaxy Zoo project successfully applied a crowdsourcing strategy, inviting online users to classify images by answering a series of questions. Unfortunately, even this approach does not scale well enough to keep up with the increasing availability of galaxy images.We present a deep neural network model for galaxy morphology classification which exploits translational and rotational symmetry. It was developed in the context of the Galaxy Challenge, an international competition to build the best model for morphology classification based on annotated images from the Galaxy Zoo project.For images with high agreement among the Galaxy Zoo participants, our model is able to reproduce their consensus with near-perfect accuracy (> 99%) for most questions. Confident model predictions are highly accurate, which makes the model suitable for filtering large collections of images and forwarding challenging images to experts for manual annotation. This approach greatly reduces the experts' workload without affecting accuracy. The application of these algorithms to larger sets of training data will be critical for analysing results from future surveys such as the LSST.

show abstract

Section: Trainingmentioning

confidence: 99%

Rotation-invariant convolutional neural networks for galaxy morphology prediction

Dieleman¹,

Willett²,

Dambre³

2015

Monthly Notices of the Royal Astronomical Society

728

604

View full text Add to dashboard Cite

show abstract

“…To evaluate the feature extraction and fusion capability of the proposed CNN, we compare it with some widely cited methods including support vector machine (SVM) [20], Adaptive Boosting technique [21], polynomial kernel PCA (KPCA) [22], sparse representation based on monogenic signal (MSRC) [23] and Iterative graph thickening model of image representations (IGT) [24]. The recognition accuracies of these algorithms are shown in Table VI.…”

Section: 4comparison With Previous Methodsmentioning

confidence: 99%

“…Accuracy (%) SVM [20] 86.73 Adaptive Boosting [21] 92.70 KPCA [22] 92.67 MSRC [23] 93.66 IGT [24] 95.00 CNN with data augmentation [25] It is desirable that the feature fusion CNNs show the ability of classifying ten-class targets regardless of the existence of variants. Meanwhile, neither increasing the width of network nor the complexity of architectures, the excellent combination of high-level feature representations and learning of potential invariant feature can achieve particularly good performance even when the labelled training data is limited.…”

Section: Methodsmentioning

confidence: 99%

Feature Fusion Based on Convolutional Neural Network for SAR ATR

Chen

Zhan

et al. 2017

ITM Web Conf.

View full text Add to dashboard Cite

Abstract-Recent breakthroughs in algorithms related to deep convolutional neural networks (DCNN) have stimulated the development of various of signal processing approaches, where the specific application of Automatic Target Recognition (ATR) using Synthetic Aperture Radar (SAR) data has spurred widely attention. Inspired by the more efficient distributed training such as inception architecture and residual network, a new feature fusion structure which jointly exploits all the merits of each version is proposed to reduce the data dimensions and the complexity of computation. The detailed procedure presented in this paper consists of the fused features, which make the representation of SAR images more distinguishable after the extraction of a set of features from DCNN, followed by a trainable classifier. In particular, the obtained results on the 10-class benchmark data set demonstrate that the presented architecture can achieve remarkable classification performance to the current state-of-the-art methods.

show abstract

“…The proposed deep learning method is expected to solve the feature automatic design problem. [13][14][15][16][17] Deep learning is a bio-inspired architecture which describes data such as image, voice and text by mimicking the human brain mechanisms of learning and analysis. Through deep learning, features are transformed from original space of lower layer to new space in higher layer.…”

Section: Introductionmentioning

confidence: 99%

Pedestrian detection algorithm in traffic scene based on weakly supervised hierarchical deep model

Cai

Wang

et al. 2016

International Journal of Advanced Robotic Systems

View full text Add to dashboard Cite

The emergence and development of deep learning theory in machine learning field provide new method for visual-based pedestrian recognition technology. To achieve better performance in this application, an improved weakly supervised hierarchical deep learning pedestrian recognition algorithm with two-dimensional deep belief networks is proposed. The improvements are made by taking into consideration the weaknesses of structure and training methods of existing classifiers. First, traditional one-dimensional deep belief network is expanded to two-dimensional that allows image matrix to be loaded directly to preserve more information of a sample space. Then, a determination regularization term with small weight is added to the traditional unsupervised training objective function. By this modification, original unsupervised training is transformed to weakly supervised training. Subsequently, that gives the extracted features discrimination ability. Multiple sets of comparative experiments show that the performance of the proposed algorithm is better than other deep learning algorithms in recognition rate and outperforms most of the existing state-of-the-art methods in non-occlusion pedestrian data set while performs fair in weakly and heavily occlusion data set.

show abstract

Advances in optimizing recurrent networks

Cited by 380 publications

References 13 publications

Rotation-invariant convolutional neural networks for galaxy morphology prediction

Rotation-invariant convolutional neural networks for galaxy morphology prediction

Feature Fusion Based on Convolutional Neural Network for SAR ATR

Pedestrian detection algorithm in traffic scene based on weakly supervised hierarchical deep model

Contact Info

Product

Resources

About