Multi-column deep neural networks for image classification

Cireşan, Dan; Meier, Ueli; Schmidhuber, Juergen

doi:10.1109/cvpr.2012.6248110

Cited by 2,889 publications

(1,821 citation statements)

References 25 publications

Supporting

Mentioning

1,790

Contrasting

Unclassified

Order By: Relevance

“…An ensemble of GPU-MPCNNs was also the first method to achieve human-competitive performance (around 0.2%) on MNIST (Ciresan et al, 2012c). This represented a dramatic improvement, since by then the MNIST record had hovered around 0.4% for almost a decade (Sec.…”

Section: : Mpcnns On Gpu Achieve Superhuman Vision Performancementioning

confidence: 99%

Deep learning in neural networks: An overview

2015

View full text Add to dashboard Cite

In recent years, deep artificial neural networks (including recurrent ones) have won numerous contests in pattern recognition and machine learning. This historical survey compactly summarises relevant work, much of it from the previous millennium. Shallow and deep learners are distinguished by the depth of their credit assignment paths, which are chains of possibly learnable, causal links between actions and effects. I review deep supervised learning (also recapitulating the history of backpropagation), unsupervised learning, reinforcement learning & evolutionary computation, and indirect search for short programs encoding deep and large networks.LATEX source: http://www.idsia.ch/˜juergen/DeepLearning8Oct2014.tex Complete BIBTEX file (888 kB): http://www.idsia.ch/˜juergen/deep.bib Preface This is the preprint of an invited Deep Learning (DL) overview. One of its goals is to assign credit to those who contributed to the present state of the art. I acknowledge the limitations of attempting to achieve this goal. The DL research community itself may be viewed as a continually evolving, deep network of scientists who have influenced each other in complex ways. Starting from recent DL results, I tried to trace back the origins of relevant ideas through the past half century and beyond, sometimes using "local search" to follow citations of citations backwards in time. Since not all DL publications properly acknowledge earlier relevant work, additional global search strategies were employed, aided by consulting numerous neural network experts. As a result, the present preprint mostly consists of references. Nevertheless, through an expert selection bias I may have missed important work. A related bias was surely introduced by my special familiarity with the work of my own DL research group in the past quarter-century. For these reasons, this work should be viewed as merely a snapshot of an ongoing credit assignment process. To help improve it, please do not hesitate to send corrections and suggestions to juergen@idsia.ch.

show abstract

Section: : Mpcnns On Gpu Achieve Superhuman Vision Performancementioning

confidence: 99%

Deep learning in neural networks: An overview

2015

View full text Add to dashboard Cite

show abstract

“…For example, it is unclear what in the images the learned networks actually look at and how the input image is represented in them. This is in stark contrast with the recent accelerated improvements of methods for training deep networks [2,8,13,6]. This lack of understanding leads to real problems; for example, a lot of trial-and-errors are necessary when designing the network architecture for each problem.…”

Section: Introductionmentioning

confidence: 49%

Understanding Convolutional Neural Networks in Terms of Category-Level Attributes

Ozeki

Okatani

2015

Computer Vision -- ACCV 2014

View full text Add to dashboard Cite

Abstract. It has been recently reported that convolutional neural networks (CNNs) show good performances in many image recognition tasks. They significantly outperform the previous approaches that are not based on neural networks particularly for object category recognition. These performances are arguably owing to their ability of discovering better image features for recognition tasks through learning, resulting in the acquisition of better internal representations of the inputs. However, in spite of the good performances, it remains an open question why CNNs work so well and/or how they can learn such good representations. In this study, we conjecture that the learned representation can be interpreted as category-level attributes that have good properties. We conducted several experiments by using the dataset AwA (Animals with Attributes) and a CNN trained for ILSVRC-2012 in a fully supervised setting to examine this conjecture. We report that there exist units in the CNN that can predict some of the 85 semantic attributes fairly accurately, along with a detailed observation that this is true only for visual attributes and not for non-visual ones. It is more natural to think that the CNN may discover not only semantic attributes but non-semantic ones (or ones that are difficult to represent as a word). To explore this possibility, we perform zero-shot learning by regarding the activation pattern of upper layers as attributes describing the categories. The result shows that it outperforms the state-of-the-art with a significant margin.

show abstract

“…Correct identification of a digit '0' was highest at p = 0.985 and correct identification of a '5' lowest at p = 0.874. The five most significant error confusions are (2,8), (9,4), (4,9), (5,8), (5,3).…”

Section: Implementation and Resultsmentioning

confidence: 99%

“…2 The Convnet chosen uses a hierarchy of three macro-levels, each level comprises a convolutional layer, rectified linear unit layer, max pool layer, and drop out layer. At the top of all this, there is an output processing layer termed 'softmax' or normalised exponential, making 13 layers in total.…”

Section: Performance Using a Deep Network And Independent Data Sourcesmentioning

confidence: 99%

Learning to Shape Errors with a Confusion Objective

Scholz

2018

Foundations of Trusted Autonomy

View full text Add to dashboard Cite

Multi-column deep neural networks for image classification

Cited by 2,889 publications

References 25 publications

Deep learning in neural networks: An overview

Deep learning in neural networks: An overview

Understanding Convolutional Neural Networks in Terms of Category-Level Attributes

Learning to Shape Errors with a Confusion Objective

Contact Info

Product

Resources

About