StarNet: towards Weakly Supervised Few-Shot Object Detection

Karlinsky, Leonid; Shtok, Joseph; Alfassy, Amit; Lichtenstein, Moshe; Harary, Sivan; Schwartz, Eli; Doveh, Sivan; Sattigeri, Prasanna; Feris, Rogério; Bronstein, Alexander M.; Giryes, Raja

doi:10.48550/arxiv.2003.06798

Cited by 3 publications

(4 citation statements)

References 37 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To our best knowledge, instead of designing a distinctly new detection framework from scratch, FSOD appends the process of meta knowledge extraction and sharing on classic deep learning object detection baselines, such as two-stage Faster R-CNN [11,16,17,72,73,78,[83][84][85][86][87][88][89][90][91][92][93][94][95][96][97][98], one-stage YOLO [10,74,99], CenterNet [100,101], and Vision Transformer [102]. The numerous published reports indicate that the two-stage network is more favored, and the two-stage network has more advantages because of its higher detection accuracy, more interpretive, and extensible network structure.…”

Section: Model: Semantics Extraction and Cross-domain Mappingmentioning

confidence: 99%

Few-Shot Object Detection in Remote Sensing Image Interpretation: Opportunities and Challenges

Liu

You

et al. 2022

Remote Sensing

View full text Add to dashboard Cite

Recent years have witnessed rapid development and remarkable achievements on deep learning object detection in remote sensing (RS) images. The growing improvement of the accuracy is inseparable from the increasingly complex deep convolutional neural network and the huge amount of sample data. However, the under-fitting neural network will damage the detection performance facing the difficulty of sample acquisition. Thus, it evolves into few-shot object detection (FSOD). In this article, we first briefly introduce the object detection task and its algorithms, to better understand the basic detection frameworks followed by FSOD. Then, FSOD design methods in RS images for three important aspects, such as sample, model, and learning strategy, are respectively discussed. In addition, some valuable research results of FSOD in computer vision field are also included. We advocate a wide research technique route, and some advice about feature enhancement and multi-modal fusion, semantics extraction and cross-domain mapping, fine-tune and meta-learning strategies, and so on, are provided. Based on our stated research route, a novel few-shot detector that focuses on contextual information is proposed. At the end of the paper, we summarize accuracy performance on experimental datasets to illustrate the achievements and shortcomings of the stated algorithms, and highlight the future opportunities and challenges of FSOD in RS image interpretation, in the hope of providing insights into future research.

show abstract

Section: Model: Semantics Extraction and Cross-domain Mappingmentioning

confidence: 99%

Few-Shot Object Detection in Remote Sensing Image Interpretation: Opportunities and Challenges

Liu

You

et al. 2022

Remote Sensing

View full text Add to dashboard Cite

show abstract

“…A few works [36,11,16] have tried XAI for FSL tasks. Geng et al [11] uses a knowledge graph to make an explanation for zero-shot tasks.…”

Section: Explainable Aimentioning

confidence: 99%

“…Sun et al [36] adopts layerwise relevance propagation (LRP) [1] to explain the output of a classifier. StarNet [16] realize visualization through heat maps derived from back-project. Recently, a new type of XAI, coined SCOUTER [20], has been proposed, which applies the self-attention mechanism [38] to the classifier.…”

Section: Explainable Aimentioning

confidence: 99%

Match Them Up: Visually Explainable Few-shot Image Classification

Wang¹,

Li²,

Verma³

et al. 2020

Preprint

View full text Add to dashboard Cite

Few-shot learning (FSL) approaches are usually based on an assumption that the pre-trained knowledge can be obtained from base (seen) categories and can be well transferred to novel (unseen) categories. However, there is no guarantee, especially for the latter part. This issue leads to the unknown nature of the inference process in most FSL methods, which hampers its application in some risksensitive areas. In this paper, we reveal a new way to perform FSL for image classification, using visual representations from the backbone model and weights generated by a newly-emerged explainable classifier. The weighted representations only include a minimum number of distinguishable features and the visualized weights can serve as an informative hint for the FSL process. Finally, a discriminator will compare the representations of each pair of the images in the support set and the query set. Pairs with the highest scores will decide the classification results. Experimental results prove that the proposed method can achieve both good accuracy and satisfactory explainability on three mainstream datasets. Code is available 6 .

show abstract

“…Enhancements to Prototypical Networks (ProtoNets) [29] in particular have been proposed for difficult few-shot tasks involving, for instance, inhomogeneous noisy datasets [8], domain adaptation [24], and relation classification in text [10]. Several works have incorporated localization conditioning into a few-shot classification architecture [12,17,18,26,32]. We include comparison with the Few-Shot Localization (FSL) method presented in [32] in our experimental evaluations.…”

Section: Introductionmentioning

confidence: 99%

Prototypical Region Proposal Networks for Few-Shot Localization and Classification

Skomski,

Tuor,

Avila

et al. 2021

Preprint

View full text Add to dashboard Cite

Recently proposed few-shot image classification methods have generally focused on use cases where the objects to be classified are the central subject of images. Despite success on benchmark vision datasets aligned with this use case, these methods typically fail on use cases involving densely-annotated, busy images: images common in the wild where objects of relevance are not the central subject, instead appearing potentially occluded, small, or among other incidental objects belonging to other classes of potential interest. To localize relevant objects, we employ a prototype-based few-shot segmentation model which compares the encoded features of unlabeled query images with support class centroids to produce region proposals indicating the presence and location of support set classes in a query image. These region proposals are then used as additional conditioning input to few-shot image classifiers. We develop a framework to unify the two stages (segmentation and classification) into an end-to-end classification model-PRoPnet-and empirically demonstrate that our methods improve accuracy on image datasets with natural scenes containing multiple object classes.

show abstract

StarNet: towards Weakly Supervised Few-Shot Object Detection

Cited by 3 publications

References 37 publications

Few-Shot Object Detection in Remote Sensing Image Interpretation: Opportunities and Challenges

Few-Shot Object Detection in Remote Sensing Image Interpretation: Opportunities and Challenges

Match Them Up: Visually Explainable Few-shot Image Classification

Prototypical Region Proposal Networks for Few-Shot Localization and Classification

Contact Info

Product

Resources

About