STNet: Selective Tuning of Convolutional Networks for Object Localization

Biparva, Mahdi; Tsotsos, John K.

doi:10.1109/iccvw.2017.319

Cited by 15 publications

(19 citation statements)

References 36 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Much of the recent progress in image categorization has been driven by the inclusion of trainable attention modules in state‐of‐the‐art DCNN architectures. While biology is sometimes mentioned as a source of inspiration, 111–117 the attentional mechanisms that have been considered remain quite limited in comparison with the rich and diverse array of processes used by the human visual system (see Ref. 118 for a review).…”

Section: The Role Of Recurrence Beyond Recognitionmentioning

confidence: 99%

Beyond the feedforward sweep: feedback computations in the visual cortex

Kreiman

Serre

2020

Annals of the New York Academy of Sciences

View full text Add to dashboard Cite

Visual perception involves the rapid formation of a coarse image representation at the onset of visual processing, which is iteratively refined by late computational processes. These early versus late time windows approximately map onto feedforward and feedback processes, respectively. State‐of‐the‐art convolutional neural networks, the main engine behind recent machine vision successes, are feedforward architectures. Their successes and limitations provide critical information regarding which visual tasks can be solved by purely feedforward processes and which require feedback mechanisms. We provide an overview of recent work in cognitive neuroscience and machine vision that highlights the possible role of feedback processes for both visual recognition and beyond. We conclude by discussing important open questions for future research.

show abstract

Section: The Role Of Recurrence Beyond Recognitionmentioning

confidence: 99%

Beyond the feedforward sweep: feedback computations in the visual cortex

Kreiman

Serre

2020

Annals of the New York Academy of Sciences

View full text Add to dashboard Cite

show abstract

“…The gating information flows from the top layers to the intermediate and early layers in the TD pass. It is shown in STNet model [2] that the TD gating activities are sufficiently representative for object localization, and we hypothesize that they are reliable to select features for object segmentation. We experimentally support the hypothesis and show that the gating activities indeed improve the segmentation accuracy over the baseline model without a similar gating mechanism.…”

Section: Top-down Selectionmentioning

confidence: 85%

“…We follow STNet [2] formulation for the TD selection pass. The TD pass begins from the elements d i , which are set to one by the initialization module.…”

Section: Top-down Selectionmentioning

confidence: 99%

“…Selective Tuning [27,26] is a computational model of visual attention, and is an early attempt to establish the applicability of a TD selection pass along with a BU feedforward pass for basic visual tasks in dynamical networks. Selective Tuning of convolutional neural networks (STNet) [2] has proposed to formulate a neural network framework with a selective TD mechanism and is evaluated for the object localization task. STNet validates the effective role of the TD selection pass to localize relevant regions covering the object features.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Selective Segmentation Networks Using Top-Down Attention

Biparva¹,

Tsotsos²

2020

Preprint

Self Cite

View full text Add to dashboard Cite

Convolutional neural networks model the transformation of the input sensory data at the bottom of a network hierarchy to the semantic information at the top of the visual hierarchy. Feedforward processing is sufficient for some object recognition tasks. Top-Down selection is potentially required in addition to the Bottom-Up feedforward pass. It can, in part, address the shortcoming of the loss of location information imposed by the hierarchical feature pyramids. We propose a unified 2-pass framework for object segmentation that augments Bottom-Up convolutional neural networks with a Top-Down selection network. We utilize the top-down selection gating activities to modulate the bottomup hidden activities for segmentation predictions. We develop an end-to-end multi-task framework with loss terms satisfying task requirements at the two ends of the network. We evaluate the proposed network on benchmark datasets for semantic segmentation, and show that networks with the Top-Down selection capability outperform the baseline model. Additionally, we shed light on the superior aspects of the new segmentation paradigm and qualitatively and quantitatively support the efficiency of the novel framework over the baseline model that relies purely on parametric skip connections.

show abstract

“…Attention mechanism has been largely studied in Neuroscience and Computational Neuroscience [4], [5], which also has a large impact on neural computation as we need to select the most pertinent piece of information, rather than using all available information at once [6]. Motivated by the properties of attention mechanism, many researchers have applied the similar attention mechanism to several computer vision tasks and have achieved great results, including image caption generation [7], object detection [6], object tracking [8], and object localization [9]. Inspired by the above results, several frameworks are proposed for attention-based visual search with neural networks.…”

mentioning

confidence: 99%

FPAN: Fine-grained and progressive attention localization network for data retrieval

et al. 2018

View full text Add to dashboard Cite

The Localization of the target object for data retrieval is a key issue in the Intelligent and Connected Transportation Systems (ICTS). However, due to lack of intelligence in the traditional transportation system, it can take tremendous resources to manually retrieve and locate the queried objects among a large number of images. In order to solve this issue, we propose an effective method to query-based object localization that uses artificial intelligence techniques to automatically locate the queried object in the complex background. The presented method is termed as Fine-grained and Progressive Attention Localization Network (FPAN), which uses an image and a queried object as input to accurately locate the target object in the image. Specifically, the fine-grained attention module is naturally embedded into each layer of the convolution neural network (CNN), thereby gradually suppressing the regions that are irrelevant to the queried object and eventually shrinking attention to the target area.We further employ top-down attentions fusion algorithm operated by a learnable cascade up-sampling structure to establish the connection between the attention map and the exact location of the queried object in the original image.Furthermore, the FPAN is trained by multi-task learning with box segmentation loss and cosine loss. At last, we conduct comprehensive experiments on both queried-based digit localization and object tracking with synthetic and benchmark datasets, respectively. The experimental results show that our algorithm is far superior to other algorithms in the synthesis datasets and outperforms most existing trackers on the OTB and VOT datasets. KEYWORDSQuery-based object localization, fine-grained attention, progressive attention, unified framework, fully convolution neural network IntroductionWith the development of the transportation system connection network, the capacity of the data center is rapidly increasing, making efficient data retrieval an urgent need. However, due to the lack of intelligence, many of the retrieval tasks in the Intelligent and Connected Transportation Systems (ICTS) need to be done manually, resulting in inefficiencies and waste of resources. In all these data retrieval tasks in ICTS, the stable and effective queried object localization is the key task. As shown in Fig. 1, this well-known object localization by a query is of substantial importance for a wide range of applications in data retrieval such as intelligent monitoring, tracking, and vehicle positioning. However, the lack of intelligence and accuracy have restricted the development of data retireval system. In order to solve these problems, we focus on the intelligent localization technology, as shown in Fig. 1.An important question about any object localization tasks is how to efficiently match the queried template in the image under arbitrary deformation of the target object [1]. The main challenge is to filter out the background and select some regions while learning the discrimination feature space and effective d...

show abstract

STNet: Selective Tuning of Convolutional Networks for Object Localization

Cited by 15 publications

References 36 publications

Beyond the feedforward sweep: feedback computations in the visual cortex

Beyond the feedforward sweep: feedback computations in the visual cortex

Selective Segmentation Networks Using Top-Down Attention

FPAN: Fine-grained and progressive attention localization network for data retrieval

Contact Info

Product

Resources

About