The functional role of cue-driven feature-based feedback in object recognition

Thorat, Sushrut; Gerven, Marcel van; Peelen, Marius V.

doi:10.32470/ccn.2018.1044-0

Cited by 5 publications

(9 citation statements)

References 7 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Our contribution and Thorat et al (2018) are very different. For example, we use a DCNN as a model of visual processing, whereas Thorat et al (2018) used a shallow fully connected network. DCNN use is important due to its proposed relation to the ventral visual stream and their performance qualities.…”

Section: Attention In Deep Learningmentioning

confidence: 71%

“…In other words, unlike our model that has a generic attention tuning for images of the same class, their network reconfiguration is specific to each input image. Thorat et al (2018) studied the impact of a cue-based feedback approach on visual processing systems via manipulating the neural and representational capacity of the network. Our contribution and Thorat et al (2018) are very different.…”

Section: Attention In Deep Learningmentioning

confidence: 99%

“…Thorat et al (2018) studied the impact of a cue-based feedback approach on visual processing systems via manipulating the neural and representational capacity of the network. Our contribution and Thorat et al (2018) are very different. For example, we use a DCNN as a model of visual processing, whereas Thorat et al (2018) used a shallow fully connected network.…”

Section: Attention In Deep Learningmentioning

confidence: 99%

See 2 more Smart Citations

The Costs and Benefits of Goal-Directed Attention in Deep Convolutional Neural Networks

2021

View full text Add to dashboard Cite

People deploy top-down, goal-directed attention to accomplish tasks, such as finding lost keys. By tuning the visual system to relevant information sources, object recognition can become more efficient (a benefit) and more biased toward the target (a potential cost). Motivated by selective attention in categorisation models, we developed a goal-directed attention mechanism that can process naturalistic (photographic) stimuli. Our attention mechanism can be incorporated into any existing deep convolutional neural networks (DCNNs). The processing stages in DCNNs have been related to ventral visual stream. In that light, our attentional mechanism incorporates top-down influences from prefrontal cortex (PFC) to support goal-directed behaviour. Akin to how attention weights in categorisation models warp representational spaces, we introduce a layer of attention weights to the mid-level of a DCNN that amplify or attenuate activity to further a goal. We evaluated the attentional mechanism using photographic stimuli, varying the attentional target. We found that increasing goal-directed attention has benefits (increasing hit rates) and costs (increasing false alarm rates). At a moderate level, attention improves sensitivity (i.e. increases $d^{\prime }$ d ′ ) at only a moderate increase in bias for tasks involving standard images, blended images and natural adversarial images chosen to fool DCNNs. These results suggest that goal-directed attention can reconfigure general-purpose DCNNs to better suit the current task goal, much like PFC modulates activity along the ventral stream. In addition to being more parsimonious and brain consistent, the mid-level attention approach performed better than a standard machine learning approach for transfer learning, namely retraining the final network layer to accommodate the new task.

show abstract

Section: Attention In Deep Learningmentioning

confidence: 71%

Section: Attention In Deep Learningmentioning

confidence: 99%

Section: Attention In Deep Learningmentioning

confidence: 99%

See 1 more Smart Citation

The Costs and Benefits of Goal-Directed Attention in Deep Convolutional Neural Networks

2021

View full text Add to dashboard Cite

show abstract

“…Applying the neuro-modulatory effects of attention to the units in a CNN was shown to increase performance in these networks as well, more so when applied at later rather than earlier layers [88]. This use of task-performing models has also led to better theories of how attention can work than those stemming solely from neural data [103].…”

Section: Exploring Cognitive Tasksmentioning

confidence: 99%

“…Connections from frontal and parietal areas are believed to implement goal-directed selective attention. Such feedback has been added to network models to implement cued detection tasks [103,134]. Feedback from higher visual areas back to lower ones are thought to implement more immediate and general image processing such as denoising.…”

Section: Alternative Architecturesmentioning

confidence: 99%

Convolutional Neural Networks as a Model of the Visual System: Past, Present, and Future

Lindsay

2021

Journal of Cognitive Neuroscience

407

241

View full text Add to dashboard Cite

Convolutional neural networks (CNNs) were inspired by early findings in the study of biological vision. They have since become successful tools in computer vision and state-of-the-art models of both neural activity and behavior on visual tasks. This review highlights what, in the context of CNNs, it means to be a good model in computational neuroscience and the various ways models can provide insight. Specifically, it covers the origins of CNNs and the methods by which we validate them as models of biological vision. It then goes on to elaborate on what we can learn about biological vision by understanding and experimenting on CNNs and discusses emerging opportunities for the use of CNNS in vision research beyond basic object recognition. This is the author's final version. This article has been accepted for publication in the Journal ofCognitive Neuroscience.

show abstract

Modulatory feedback determines attentional object segmentation in a model of the ventral stream

Papale

Williford

Balk

et al. 2023

Preprint

View full text Add to dashboard Cite

Studies in neuroscience inspired progress in the design of artificial neural networks (ANNs) and vice versa ANNs have also started to provide new insights into the functioning of brain circuits. So far, the focus has been on how ANNs can help to explain the tuning of neurons at various stages of the visual cortical hierarchy. However, the role of modulatory feedback connections, which play a role in attention and perceptual organization, has not been tested yet. Here, we present a biologically plausible neural network that performs scene segmentation and can shift attention using modulatory feedback connections from higher to lower brain areas. The model replicates several neurophysiological signatures of recurrent processing. Specifically, figural regions elicit more neuronal activity in model units than the background. The modulation of neuronal activity by figure and ground occurs at a delay after the first feedforward response, because it depends on a loop through the higher model areas. Importantly, the enhancement of the figural response is controlled by object-based attention, which stays focused on the figural regions and does not spill over to the adjacent background, as observed in the visual cortex. Our results indicate how progress in deep learning can be used to garner insight into the recurrent cortical processing for scene segmentation and object-based attention.

show abstract

The functional role of cue-driven feature-based feedback in object recognition

Cited by 5 publications

References 7 publications

The Costs and Benefits of Goal-Directed Attention in Deep Convolutional Neural Networks

The Costs and Benefits of Goal-Directed Attention in Deep Convolutional Neural Networks

Convolutional Neural Networks as a Model of the Visual System: Past, Present, and Future

Modulatory feedback determines attentional object segmentation in a model of the ventral stream

Contact Info

Product

Resources

About