Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification

He, Kai; Zhang, Xiangyu; Ren, Shouxin; Sun, Jian

doi:10.1109/iccv.2015.123

Cited by 15,592 publications

(9,748 citation statements)

References 34 publications

Supporting

Mentioning

9,063

Contrasting

Unclassified

Order By: Relevance

“…Recent state-of-the-art neural networks have increased depth, ranging from 16 (Simonyan & Zisserman, 2014) to 152 (He, Zhang, Ren, & Sun, 2015) layers (combined with some architectural advances). While the brain is clearly not shallow, its depth is limited to substantially fewer computational layers considering feed-forward processing (Lamme & Roelfsema, 2000).…”

Section: Box 1 Deep Neural Networkmentioning

confidence: 99%

Visual pathways from the perspective of cost functions and multi-task deep neural networks

Scholte

Losch

Ramakrishnan

et al. 2018

Cortex

View full text Add to dashboard Cite

Deep learning Cost functions RepresentationsVisual processing a b s t r a c t Vision research has been shaped by the seminal insight that we can understand the higher-tier visual cortex from the perspective of multiple functional pathways with different goals. In this paper, we try to give a computational account of the functional organization of this system by reasoning from the perspective of multi-task deep neural networks. Machine learning has shown that tasks become easier to solve when they are decomposed into subtasks with their own cost function. We hypothesize that the visual system optimizes multiple cost functions of unrelated tasks and this causes the emergence of a ventral pathway dedicated to vision for perception, and a dorsal pathway dedicated to vision for action.To evaluate the functional organization in multi-task deep neural networks, we propose a method that measures the contribution of a unit towards each task, applying it to two networks that have been trained on either two related or two unrelated tasks, using an identical stimulus set. Results show that the network trained on the unrelated tasks shows a decreasing degree of feature representation sharing towards higher-tier layers while the network trained on related tasks uniformly shows high degree of sharing.We conjecture that the method we propose can be used to analyze the anatomical and functional organization of the visual system and beyond. We predict that the degree to which tasks are related is a good descriptor of the degree to which they share downstream cortical-units.

show abstract

Section: Box 1 Deep Neural Networkmentioning

confidence: 99%

Visual pathways from the perspective of cost functions and multi-task deep neural networks

Scholte

Losch

Ramakrishnan

et al. 2018

Cortex

View full text Add to dashboard Cite

show abstract

“…We selected PReLU as the activation function [20]. It introduces a very small number of parameters on the basis of the ReLU [21].…”

Section: Going Faster and Preventing Overfittingmentioning

confidence: 99%

“…From the possible batch sizes of (16,20,24,32,64), 32 was selected in view of the performance of our graphics processing unit (GPU) and test accuracy. The time limit of the initial training epochs was 400, but on the basis of our prior experimental results the best precision was reached within 80 epochs, so the number of epochs used during training was only 80.…”

Section: Experimental Setup For Classification Of Labeled Pixelsmentioning

confidence: 99%

A Fast Dense Spectral–Spatial Convolution Network Framework for Hyperspectral Images Classification

et al. 2018

View full text Add to dashboard Cite

Recent research shows that deep-learning-derived methods based on a deep convolutional neural network have high accuracy when applied to hyperspectral image (HSI) classification, but long training times. To reduce the training time and improve accuracy, in this paper we propose an end-to-end fast dense spectral-spatial convolution (FDSSC) framework for HSI classification. The FDSSC framework uses different convolutional kernel sizes to extract spectral and spatial features separately, and the "valid" convolution method to reduce the high dimensions. Densely-connected structures-the input of each convolution consisting of the output of all previous convolution layers-was used for deep learning of features, leading to extremely accurate classification. To increase speed and prevent overfitting, the FDSSC framework uses a dynamic learning rate, parametric rectified linear units, batch normalization, and dropout layers. These attributes enable the FDSSC framework to achieve accuracy within as few as 80 epochs. The experimental results show that with the Indian Pines, Kennedy Space Center, and University of Pavia datasets, the proposed FDSSC framework achieved state-of-the-art performance compared with existing deep-learning-based methods while significantly reducing the training time.

show abstract

“…The reported results for visual concept classification [9,12,15,23] indicate that human performance is (far) better than the respective state of the art for automated visual concept classification at that time. Although He et al claimed in 2015 that human-level performance has been surpassed by their approach [8], this claim remains questionable since only two human annotators were involved in the underlying study of Russakovsky et al [18]. There are also some methodological issues in the reported experiments of the other studies, for example, experimental settings are not well defined, the employment of crowdsourcing is critical, or the number of images is too small which prevents drawing conclusions for rare classes.…”

Section: Human and Machine Performance In Visual And Auditory Recognimentioning

confidence: 57%

“…Russakovsky et al compared this submission with two human annotators and discovered that the neuronal network outperformed one of them. He et al [8] claimed their system to be the first one that surpassed human-level performance (5.1%) on ImageNet data by achieving an error rate of 4.94%.…”

Section: Human and Machine Performance In Visual And Auditory Recognimentioning

confidence: 99%

“Are Machines Better Than Humans in Image Tagging?” - A User Study Adds to the Puzzle

Ewerth

Springstein

Phan-Vogtmann

et al. 2017

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Abstract. "Do machines perform better than humans in visual recognition tasks?" Not so long ago, this question would have been considered even somewhat provoking and the answer would have been clear: "No". In this paper, we present a comparison of human and machine performance with respect to annotation for multimedia retrieval tasks. Going beyond recent crowdsourcing studies in this respect, we also report results of two extensive user studies. In total, 23 participants were asked to annotate more than 1000 images of a benchmark dataset, which is the most comprehensive study in the field so far. Krippendorff's α is used to measure inter-coder agreement among several coders and the results are compared with the best machine results. The study is preceded by a summary of studies which compared human and machine performance in different visual and auditory recognition tasks. We discuss the results and derive a methodology in order to compare machine performance in multimedia annotation tasks at human level. This allows us to formally answer the question whether a recognition problem can be considered as solved. Finally, we are going to answer the initial question.

show abstract

Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification

Cited by 15,592 publications

References 34 publications

Visual pathways from the perspective of cost functions and multi-task deep neural networks

Visual pathways from the perspective of cost functions and multi-task deep neural networks

A Fast Dense Spectral–Spatial Convolution Network Framework for Hyperspectral Images Classification

“Are Machines Better Than Humans in Image Tagging?” - A User Study Adds to the Puzzle

Contact Info

Product

Resources

About