Levels of Representation in a Deep Learning Model of Categorization

Guest, Olivia; Love, Bradley C.

doi:10.1101/626374

Cited by 13 publications

(14 citation statements)

References 66 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The classes have been intentionally selected to overlap with 200 classes from ImageNet. These images can reduce DCNN performance drastically by exploiting the vulnerabilities of these networks such as their colour and texture biases (Guest and Love 2019;Hu et al 2018). Although adversarial attacks have been heavily studied (Goodfellow et al 2015;Nguyen et al 2015;Song et al 2018), these works use synthetic or unrealistic images that are carefully designed to defeat advanced DCNNs.…”

Section: Experiments 3: Natural Adversarial Imagesmentioning

confidence: 99%

The Costs and Benefits of Goal-Directed Attention in Deep Convolutional Neural Networks

2021

Self Cite

View full text Add to dashboard Cite

People deploy top-down, goal-directed attention to accomplish tasks, such as finding lost keys. By tuning the visual system to relevant information sources, object recognition can become more efficient (a benefit) and more biased toward the target (a potential cost). Motivated by selective attention in categorisation models, we developed a goal-directed attention mechanism that can process naturalistic (photographic) stimuli. Our attention mechanism can be incorporated into any existing deep convolutional neural networks (DCNNs). The processing stages in DCNNs have been related to ventral visual stream. In that light, our attentional mechanism incorporates top-down influences from prefrontal cortex (PFC) to support goal-directed behaviour. Akin to how attention weights in categorisation models warp representational spaces, we introduce a layer of attention weights to the mid-level of a DCNN that amplify or attenuate activity to further a goal. We evaluated the attentional mechanism using photographic stimuli, varying the attentional target. We found that increasing goal-directed attention has benefits (increasing hit rates) and costs (increasing false alarm rates). At a moderate level, attention improves sensitivity (i.e. increases $d^{\prime }$ d ′ ) at only a moderate increase in bias for tasks involving standard images, blended images and natural adversarial images chosen to fool DCNNs. These results suggest that goal-directed attention can reconfigure general-purpose DCNNs to better suit the current task goal, much like PFC modulates activity along the ventral stream. In addition to being more parsimonious and brain consistent, the mid-level attention approach performed better than a standard machine learning approach for transfer learning, namely retraining the final network layer to accommodate the new task.

show abstract

Section: Experiments 3: Natural Adversarial Imagesmentioning

confidence: 99%

The Costs and Benefits of Goal-Directed Attention in Deep Convolutional Neural Networks

2021

Self Cite

View full text Add to dashboard Cite

show abstract

“…In SHJ Set 2 (from Crump et al (2013)), the images are varying geometric shapes on a black background with a green border, with the three binary features being size (large or small), shape (square or triangle), and color (black or white). Finally, in SHJ Set 3 (from Guest and Love (2019)), the three binary features are the same as SHJ Set 2 but with different values and visual appearance, with size (large or small), shape (circle or square), and color (red or blue). Additionally, for each dataset, there are six different mappings (permutations) between the abstract and image-based features.…”

Section: Methodsmentioning

confidence: 99%

“…For instance, CNNs pre-trained for image classification have shown success in predicting category typicality ratings (Lake, Zaremba, Fergus, & Gureckis, 2015) and similarity ratings from natural images (Peterson, Abbott, & Griffiths, 2018). More recently, researchers have begun to combine CNNs with classic prototype and exemplar models of categorization Guest & Love, 2019;Singh, Peterson, Battleday, & Griffiths, 2020;Nosofsky, Meagher, & Kumar, 2020), usually with the aim of predicting human categorization decisions for images of common categories such as animals and vehicles.…”

Section: Introductionmentioning

confidence: 99%

Modeling artiﬁcial category learning from pixels: Revisiting Shepard, Hovland, and Jenkins (1961) with deep neural networks

Vong¹,

Lake²

2021

Preprint

View full text Add to dashboard Cite

Recent work has paired classic category learning models with convolutional neural networks (CNNs), allowing researchers to study categorization behavior from raw image inputs. However, this research typically uses naturalistic images, which assess participant responses to existing categories; yet, much of traditional category learning research has focused on using novel, artiﬁcial stimuli to examine the learning process behind how people acquire categories. In this work, we pair a CNN with ALCOVE (Kruschke, 1992), a well-known exemplar model of categorization, and attempt to examine whether this model can reproduce the classic type ordering effect from Shepard, Hovland, and Jenkins (1961) on raw images rather than abstract features. We examine this question with a variety of CNN architectures and image datasets and compare ALCOVE-CNN to two other models that lacked certain key features of ALCOVE. We found that our ALCOVE-CNN model could reproduce the type ordering effect more often than the other models we tested, but in limited situations. Our results showed that success varied greatly across the various conﬁgurations we tested, suggesting that the feature representations from CNNs provide strong constraints in properly capturing this effect.

show abstract

“…Previous researchers have used various mapping methods to gain insight into neural network activity based on the activation of network layers. For instance, previous work has used stimulus-decoding analyses and activation similarity to probe for features represented by networks and to gain insight into the processing stages corresponding to layers of the networks (e.g., Ettinger, Elgohary & Resnik, 2016 ; Qian, Qiu & Huang, 2016 ; Hupkes, Veldhoen & Zuidema, 2018 ; Guest & Love, 2019 ; Lakretz et al, 2019 ; Tenney, Das & Pavlick, 2019 ).…”

Section: Related Workmentioning

confidence: 99%

Learning to perform role-filler binding with schematic knowledge

Chen

Beukers

et al. 2021

PeerJ

View full text Add to dashboard Cite

Through specific experiences, humans learn the relationships that underlie the structure of events in the world. Schema theory suggests that we organize this information in mental frameworks called “schemata,” which represent our knowledge of the structure of the world. Generalizing knowledge of structural relationships to new situations requires role-filler binding, the ability to associate specific “fillers” with abstract “roles.” For instance, when we hear the sentence Alice ordered a tea from Bob, the role-filler bindings customer:Alice, drink:tea and barista:Bob allow us to understand and make inferences about the sentence. We can perform these bindings for arbitrary fillers—we understand this sentence even if we have never heard the names Alice, tea, or Bob before. In this work, we define a model as capable of performing role-filler binding if it can recall arbitrary fillers corresponding to a specified role, even when these pairings violate correlations seen during training. Previous work found that models can learn this ability when explicitly told what the roles and fillers are, or when given fillers seen during training. We show that networks with external memory learn to bind roles to arbitrary fillers, without explicitly labeled role-filler pairs. We further show that they can perform these bindings on role-filler pairs that violate correlations seen during training, while retaining knowledge of training correlations. We apply analyses inspired by neural decoding to interpret what the networks have learned.

show abstract

Levels of Representation in a Deep Learning Model of Categorization

Cited by 13 publications

References 66 publications

The Costs and Benefits of Goal-Directed Attention in Deep Convolutional Neural Networks

The Costs and Benefits of Goal-Directed Attention in Deep Convolutional Neural Networks

Modeling artiﬁcial category learning from pixels: Revisiting Shepard, Hovland, and Jenkins (1961) with deep neural networks

Learning to perform role-filler binding with schematic knowledge

Contact Info

Product

Resources

About