Human Uncertainty Makes Classification More Robust

Peterson, J.L.; Battleday, Ruairidh M.; Griffiths, Thomas L.; Russakovsky, Olga

doi:10.1109/iccv.2019.00971

Cited by 144 publications

(208 citation statements)

References 41 publications

Supporting

Mentioning

206

Contrasting

Order By: Relevance

“…One natural extension here would be to investigate the feasibility of using a machine learning dataset with lower-level category labels; for example, a subset of the ImageNet database 58 . Beyond their interest for psychology, human behavioral data for these domains provides a rich yet largely unexploited training signal for computer vision systems, as we recently demonstrated by using such information to improve the generalization performance and robustness of natural image classifiers 65 . More broadly, these results highlight the potential of a new paradigm for psychological research that draws on the increasingly abundant datasets, machine learning tools, and behavioral data available online, rather than procuring them for individual experiments at heavy computational and experimental cost.…”

Section: Discussionmentioning

confidence: 99%

Capturing human categorization of natural images by combining deep networks and cognitive models

2020

View full text Add to dashboard Cite

Human categorization is one of the most important and successful targets of cognitive modeling, with decades of model development and assessment using simple, low-dimensional artificial stimuli. However, it remains unclear how these findings relate to categorization in more natural settings, involving complex, high-dimensional stimuli. Here, we take a step towards addressing this question by modeling human categorization over a large behavioral dataset, comprising more than 500,000 judgments over 10,000 natural images from ten object categories. We apply a range of machine learning methods to generate candidate representations for these images, and show that combining rich image representations with flexible cognitive models captures human decisions best. We also find that in the high-dimensional representational spaces these methods generate, simple prototype models can perform comparably to the more complex memory-based exemplar models dominant in laboratory settings.

show abstract

Section: Discussionmentioning

confidence: 99%

Capturing human categorization of natural images by combining deep networks and cognitive models

2020

View full text Add to dashboard Cite

show abstract

“…56,123,124 A prime motive for trying to integrate cognitively motivated category structure back into CNN learning paradigms is, then, that it may help provide a solution to this new frontier of problems. Peterson et al 125 attempted to do so indirectly by training a range of CNNs to predict the human categorization judgments ("human" or "soft" labels) from the CIFAR-10H dataset. They found that these CNNs performed significantly better on a number of outof-sample natural image sets than CNNs trained on CIFAR-10, while scoring the same on the hardlabel validation set (Fig.…”

Section: Similaritymentioning

confidence: 99%

“…It appeared as though training with human labels endowed networks with more tolerance to noise and more graceful degradation: exactly the current aims of the computer vision community, and those properties originally sought by early artificial intelligence researchers. Finally, Peterson et al 125 showed that CNNs trained with human labels invariably performed better than alternative strategies, which either incorporate random label noise or train on convex combinations of image-label pairs. 126 This shows that the structure contained in human labels is helpful for classification beyond the regularization effects of adding training noise.…”

Section: Similaritymentioning

confidence: 99%

From convolutional neural networks to models of higher‐level cognition (and back again)

Battleday

Peterson

Griffiths

2021

Annals of the New York Academy of Sciences

View full text Add to dashboard Cite

The remarkable successes of convolutional neural networks (CNNs) in modern computer vision are by now well known, and they are increasingly being explored as computational models of the human visual system. In this paper, we ask whether CNNs might also provide a basis for modeling higher-level cognition, focusing on the core phenomena of similarity and categorization. The most important advance comes from the ability of CNNs to learn high-dimensional representations of complex naturalistic images, substantially extending the scope of traditional cognitive models that were previously only evaluated with simple artificial stimuli. In all cases, the most successful combinations arise when CNN representations are used with cognitive models that have the capacity to transform them to better fit human behavior. One consequence of these insights is a toolkit for the integration of cognitively motivated constraints back into CNN training paradigms in computer vision and machine learning, and we review cases where this leads to improved performance. A second consequence is a roadmap for how CNNs and cognitive models can be more fully integrated in the future, allowing for flexible end-to-end algorithms that can learn representations from data while still retaining the structured behavior characteristic of human cognition.

show abstract

“…However, to our best knowledge, very little work has been reported on the comprehensive interpretation of deep learning methods, especially for COVID-19 diagnosis. In recent years, the deep learning methods arise and have been widely used in a wide range of machine learning tasks, such as image classification [18][19][20], image segmentation [21,22], natural language processing [23,24], and etc. Due to the good performance of deep learning methods, it has been utilized for resolving the problem of automatic COVID-19 analysis by many researchers.…”

Section: Introductionmentioning

confidence: 99%

An Interpretation Architecture for Deep Learning Models with the Application of COVID-19 Diagnosis

Wan

Zhou

Zhang

2021

Entropy

View full text Add to dashboard Cite

The Coronavirus disease 2019 (COVID-19) has become one of the threats to the world. Computed tomography (CT) is an informative tool for the diagnosis of COVID-19 patients. Many deep learning approaches on CT images have been proposed and brought promising performance. However, due to the high complexity and non-transparency of deep models, the explanation of the diagnosis process is challenging, making it hard to evaluate whether such approaches are reliable. In this paper, we propose a visual interpretation architecture for the explanation of the deep learning models and apply the architecture in COVID-19 diagnosis. Our architecture designs a comprehensive interpretation about the deep model from different perspectives, including the training trends, diagnostic performance, learned features, feature extractors, the hidden layers, the support regions for diagnostic decision, and etc. With the interpretation architecture, researchers can make a comparison and explanation about the classification performance, gain insight into what the deep model learned from images, and obtain the supports for diagnostic decisions. Our deep model achieves the diagnostic result of 94.75%, 93.22%, 96.69%, 97.27%, and 91.88% in the criteria of accuracy, sensitivity, specificity, positive predictive value, and negative predictive value, which are 8.30%, 4.32%, 13.33%, 10.25%, and 6.19% higher than that of the compared traditional methods. The visualized features in 2-D and 3-D spaces provide the reasons for the superiority of our deep model. Our interpretation architecture would allow researchers to understand more about how and why deep models work, and can be used as interpretation solutions for any deep learning models based on convolutional neural network. It can also help deep learning methods to take a step forward in the clinical COVID-19 diagnosis field.

show abstract

Human Uncertainty Makes Classification More Robust

Cited by 144 publications

References 41 publications

Capturing human categorization of natural images by combining deep networks and cognitive models

Capturing human categorization of natural images by combining deep networks and cognitive models

From convolutional neural networks to models of higher‐level cognition (and back again)

An Interpretation Architecture for Deep Learning Models with the Application of COVID-19 Diagnosis

Contact Info

Product

Resources

About