Inferring the Class Conditional Response Map for Weakly Supervised Semantic Segmentation

Sun, Weixuan; Zhang, Jing; Barnes, Nick

doi:10.1109/wacv51458.2022.00271

Cited by 16 publications

(9 citation statements)

References 39 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Indeed, WSSS transforms the semantic segmentation task into the much less demanding effort of image-level class annotations. The problem is ill-conditioned and difficult, and a large literature is dedicated to the solution of it starting from Zeiler and Fergus ( 2014 ) and Zhou et al ( 2016 ), up to most recent contributions (Chang et al, 2020 ; Sun et al, 2020 , 2022 ; Wang et al, 2020 ; Wu et al, 2021 ; Zhang et al, 2021 ). Semantic segmentation is critical for detecting tip-burn on large canopies due to the difficulty of both identifying it on a dense set of plants and to individually localizing each tip-burned plant within the canopy, as shown in Figure 3 .…”

Section: Methodsmentioning

confidence: 99%

Detection and Localization of Tip-Burn on Large Lettuce Canopies

Franchetti¹,

Pirri

2022

Front. Plant Sci.

View full text Add to dashboard Cite

Recent years have seen an increased effort in the detection of plant stresses and diseases using non-invasive sensors and deep learning methods. Nonetheless, no studies have been made on dense plant canopies, due to the difficulty in automatically zooming into each plant, especially in outdoor conditions. Zooming in and zooming out is necessary to focus on the plant stress and to precisely localize the stress within the canopy, for further analysis and intervention. This work concentrates on tip-burn, which is a plant stress affecting lettuce grown in controlled environmental conditions, such as in plant factories. We present a new method for tip-burn stress detection and localization, combining both classification and self-supervised segmentation to detect, localize, and closely segment the stressed regions. Starting with images of a dense canopy collecting about 1,000 plants, the proposed method is able to zoom into the tip-burn region of a single plant, covering less than 1/10th of the plant itself. The method is crucial for solving the manual phenotyping that is required in plant factories. The precise localization of the stress within the plant, of the plant within the tray, and of the tray within the table canopy allows to automatically deliver statistics and causal annotations. We have tested our method on different data sets, which do not provide any ground truth segmentation mask, neither for the leaves nor for the stresses; therefore, the results on the self-supervised segmentation is even more impressive. Results show that the accuracy for both classification and self supervised segmentation is new and efficacious. Finally, the data set used for training test and validation is currently available on demand.

show abstract

Section: Methodsmentioning

confidence: 99%

Detection and Localization of Tip-Burn on Large Lettuce Canopies

Franchetti¹,

Pirri

2022

Front. Plant Sci.

View full text Add to dashboard Cite

show abstract

“…To address this problem, we propose a linear self-attention that is aware of 2D position. Locality is a widely used assumption in computer vision [10]- [13], [34]- [38], i.e., neighbouring pixels should have higher possibilities to belong to the same object than distant pixels. Convolution-based networks are inherently coupled with this assumption as they always exploit feature locality [39], [40].…”

Section: Self-attention Linearization Localitymentioning

confidence: 99%

Vicinity Vision Transformer

Sun¹,

Qin²,

Deng³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Vision transformers have shown great success on numerous computer vision tasks. However, its central component, softmax attention, prohibits vision transformers from scaling up to high-resolution images, due to both the computational complexity and memory footprint being quadratic. Although linear attention was introduced in natural language processing (NLP) tasks to mitigate a similar issue, directly applying existing linear attention to vision transformers may not lead to satisfactory results. We investigate this problem and find that computer vision tasks focus more on local information compared with NLP tasks. Based on this observation, we present a Vicinity Attention that introduces a locality bias to vision transformers with linear complexity. Specifically, for each image patch, we adjust its attention weight based on its 2D Manhattan distance measured by its neighbouring patches. In this case, the neighbouring patches will receive stronger attention than far away patches. Moreover, since our Vicinity Attention requires the token length to be much larger than the feature dimension to show its efficiency advantages, we further propose a new Vicinity Vision Transformer (VVT) structure to reduce the feature dimension without degenerating the accuracy. We perform extensive experiments on the CIFAR100, ImageNet1K, and ADE20K datasets to validate the effectiveness of our method. Our method has a slower growth rate of GFlops than previous transformer-based and convolution-based networks when the input resolution increases. In particular, our approach achieves state-of-the-art image classification accuracy with 50% fewer parameters than previous methods.

show abstract

“…The vision features are usually continuous in temporal [11,43,101,59] and spatial [73,62,70,18] neighbourhood. Analogously, the prediction confidence increases if all the tokens within a local region can reach a consensus for feature importance.…”

Section: Neighborhood Associationmentioning

confidence: 99%

“…Meanwhile, the salient feature activations are usually locally accumulative [11,73]. In the continuous video, critical information contained in the target token could be shared by its spatial and/or temporal neighbour tokens.…”

Section: Introductionmentioning

confidence: 99%

Linear Video Transformer with Feature Fixation

Lu¹,

Liu²,

Wang³

et al. 2022

Preprint

View full text Add to dashboard Cite

Vision Transformers have achieved impressive performance in video classification, while suffering from the quadratic complexity caused by the Softmax attention mechanism. Some studies alleviate the computational costs by reducing the number of tokens in attention calculation, but the complexity is still quadratic. Another promising way is to replace Softmax attention with linear attention, which owns linear complexity but presents a clear performance drop. We find that such a drop in linear attention results from the lack of attention concentration on critical features. Therefore, we propose a feature fixation module to reweight feature importance of the query and key before computing linear attention. Specifically, we regard the query, key, and value as various latent representations of the input token, and learn the feature fixation ratio by aggregating Query-Key-Value information. This is beneficial for measuring the feature importance comprehensively. Furthermore, we enhance the feature fixation by neighborhood association, which leverages additional guidance from spatial and temporal neighbouring tokens. The proposed method significantly improves the linear attention baseline and achieves state-of-the-art performance among linear video Transformers on three popular video classification benchmarks. With fewer parameters and higher efficiency, our performance is even comparable to some Softmax-based quadratic Transformers.

show abstract

Inferring the Class Conditional Response Map for Weakly Supervised Semantic Segmentation

Cited by 16 publications

References 39 publications

Detection and Localization of Tip-Burn on Large Lettuce Canopies

Detection and Localization of Tip-Burn on Large Lettuce Canopies

Vicinity Vision Transformer

Linear Video Transformer with Feature Fixation

Contact Info

Product

Resources

About