THINGS: A database of 1,854 object concepts and more than 26,000 naturalistic object images

Hebart, Martin N.; Dickter, Adam H.; Kidder, Alexis; Kwok, Wan Y.; Corriveau, Anna; Wicklin, Caitlin Van; Baker, Chris I.

doi:10.1371/journal.pone.0223792

Cited by 154 publications

(166 citation statements)

References 71 publications

Supporting

Mentioning

165

Contrasting

Order By: Relevance

“…First, we needed to identify a set of objects that is representative of the objects encountered in the real world. For that purpose, we chose the 1,854 objects in the THINGS database 17 , which we developed to provide a comprehensive list of living and non-living things according to their everyday use in the American English language. For each object, we chose a representative image that had been shown to be named consistently during the creation of this database.…”

Section: Resultsmentioning

confidence: 99%

Revealing the multidimensional mental representations of natural objects underlying human similarity judgements

et al. 2020

Self Cite

View full text Add to dashboard Cite

Objects can be characterized according to a vast number of possible criteria (e.g. animacy, shape, color, function), but some dimensions are more useful than others for making sense of the objects around us. To identify these “core dimensions” of object representations, we developed a data-driven computational model of similarity judgments for real-world images of 1,854 objects. The model captured most explainable variance in similarity judgments and produced 49 highly reproducible and meaningful object dimensions that reflect various conceptual and perceptual properties of those objects. These dimensions predicted external categorization behavior and reflected typicality judgments of those categories. Further, humans can accurately rate objects along these dimensions, highlighting their interpretability and opening up a way to generate similarity estimates from object dimensions alone. Collectively, these results demonstrate that human similarity judgments can be captured by a fairly low-dimensional, interpretable embedding that generalizes to external behavior.

show abstract

Section: Resultsmentioning

confidence: 99%

Revealing the multidimensional mental representations of natural objects underlying human similarity judgements

et al. 2020

Self Cite

View full text Add to dashboard Cite

show abstract

“…In addition to considering behavioural measures, model representations can be evaluated against high-quality neuroimaging studies in which participants view naturalistic images (e.g. Hebart et al 2019).…”

Section: Discussionmentioning

confidence: 99%

The Costs and Benefits of Goal-Directed Attention in Deep Convolutional Neural Networks

2021

View full text Add to dashboard Cite

People deploy top-down, goal-directed attention to accomplish tasks, such as finding lost keys. By tuning the visual system to relevant information sources, object recognition can become more efficient (a benefit) and more biased toward the target (a potential cost). Motivated by selective attention in categorisation models, we developed a goal-directed attention mechanism that can process naturalistic (photographic) stimuli. Our attention mechanism can be incorporated into any existing deep convolutional neural networks (DCNNs). The processing stages in DCNNs have been related to ventral visual stream. In that light, our attentional mechanism incorporates top-down influences from prefrontal cortex (PFC) to support goal-directed behaviour. Akin to how attention weights in categorisation models warp representational spaces, we introduce a layer of attention weights to the mid-level of a DCNN that amplify or attenuate activity to further a goal. We evaluated the attentional mechanism using photographic stimuli, varying the attentional target. We found that increasing goal-directed attention has benefits (increasing hit rates) and costs (increasing false alarm rates). At a moderate level, attention improves sensitivity (i.e. increases $d^{\prime }$ d ′ ) at only a moderate increase in bias for tasks involving standard images, blended images and natural adversarial images chosen to fool DCNNs. These results suggest that goal-directed attention can reconfigure general-purpose DCNNs to better suit the current task goal, much like PFC modulates activity along the ventral stream. In addition to being more parsimonious and brain consistent, the mid-level attention approach performed better than a standard machine learning approach for transfer learning, namely retraining the final network layer to accommodate the new task.

show abstract

“…Ecoset was created as a large-scale image resource for deep learning and human visual neuroscience more generally (see ref. 43 for a related dataset designed for experimental work in psychology and neuroscience). A total of 565 categories were selected based on the following: 1) their word frequency in American television and film subtitles (SUBTLEX_US, 10), 2) the perceived concreteness by human observers ( 11 ), and 3) the availability of a minimum of 700 images.…”

Section: Methodsmentioning

confidence: 99%

An ecologically motivated image dataset for deep learning yields better models of human vision

Mehrer

Spoerer

Jones

et al. 2021

Proc. Natl. Acad. Sci. U.S.A.

121

109

View full text Add to dashboard Cite

Deep neural networks provide the current best models of visual information processing in the primate brain. Drawing on work from computer vision, the most commonly used networks are pretrained on data from the ImageNet Large Scale Visual Recognition Challenge. This dataset comprises images from 1,000 categories, selected to provide a challenging testbed for automated visual object recognition systems. Moving beyond this common practice, we here introduce ecoset, a collection of >1.5 million images from 565 basic-level categories selected to better capture the distribution of objects relevant to humans. Ecoset categories were chosen to be both frequent in linguistic usage and concrete, thereby mirroring important physical objects in the world. We test the effects of training on this ecologically more valid dataset using multiple instances of two neural network architectures: AlexNet and vNet, a novel architecture designed to mimic the progressive increase in receptive field sizes along the human ventral stream. We show that training on ecoset leads to significant improvements in predicting representations in human higher-level visual cortex and perceptual judgments, surpassing the previous state of the art. Significant and highly consistent benefits are demonstrated for both architectures on two separate functional magnetic resonance imaging (fMRI) datasets and behavioral data, jointly covering responses to 1,292 visual stimuli from a wide variety of object categories. These results suggest that computational visual neuroscience may take better advantage of the deep learning framework by using image sets that reflect the human perceptual and cognitive experience. Ecoset and trained network models are openly available to the research community.

show abstract

THINGS: A database of 1,854 object concepts and more than 26,000 naturalistic object images

Cited by 154 publications

References 71 publications

Revealing the multidimensional mental representations of natural objects underlying human similarity judgements

Revealing the multidimensional mental representations of natural objects underlying human similarity judgements

The Costs and Benefits of Goal-Directed Attention in Deep Convolutional Neural Networks

An ecologically motivated image dataset for deep learning yields better models of human vision

Contact Info

Product

Resources

About