One-Shot Learning for Semantic Segmentation

Shaban, Amirreza; Bansal, Shray; Liu, Zhen; Essa, Irfan; Boots, Byron

doi:10.48550/arxiv.1709.03410

Cited by 97 publications

(258 citation statements)

References 0 publications

Supporting

Mentioning

257

Contrasting

Unclassified

Order By: Relevance

“…We demonstrate the effectiveness of our method on several benchmarks [27,28,54]. Although not specifically designed for semantic correspondence task, our work attains state-of-the-art performance on all the benchmarks for fewshot segmentation and achieves highly competitive results even for semantic correspondence, showing its superiority over the recently proposed methods.…”

Section: Introductionmentioning

confidence: 82%

“…Inspired by few-shot learning paradigm [48,57], which aims to learn-to-learn a model for a novel task with only a limited number of samples, fewshot segmentation has received considerable attention. Following the success of [54], prototypical networks [57] and numerous other works [8,25,30,32,43,55,59,68,[75][76][77]82] proposed to utilize a prototype extracted from support samples, which is used to refine the query features to contain the relevant support information. In addition, inspired by [80] that observed the use of high-level features leads to a performance drop, [62] proposed to utilize high-level features by computing a prior map which takes maximum score within a correlation map.…”

Section: Related Workmentioning

confidence: 99%

“…The goal of few-shot segmentation is to segment an object of unseen classes from a query image given only a few annotated examples [54]. To mitigate the overfitting caused by insufficient training data, we follow the common protocol called episodic training [66].…”

Section: Problem Formulationmentioning

confidence: 99%

“…With the advent of deep networks and the availability of large-scale datasets with ground-truth segmentation annotations, substantial progress has been made in this task [4,5,34,44,60]. However, as such advances can be attributed to abundant pixel-wise segmentation maps made by manual annotation, which is often labor-intensive, few-shot segmentation task [48,54] has been introduced to address this, where only a handful of support samples are provided to make a mask prediction for a query, which mitigates the reliance on the labeled data.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Cost Aggregation Is All You Need for Few-Shot Segmentation

Hong¹,

Cho²,

Nam³

et al. 2021

Preprint

View full text Add to dashboard Cite

We introduce a novel cost aggregation network, dubbed Volumetric Aggregation with Transformers (VAT), to tackle the few-shot segmentation task by using both convolutions and transformers to efficiently handle high dimensional correlation maps between query and support. In specific, we propose our encoder consisting of volume embedding module to not only transform the correlation maps into more tractable size but also inject some convolutional inductive bias and volumetric transformer module for the cost aggregation. Our encoder has a pyramidal structure to let the coarser level aggregation to guide the finer level and enforce to learn complementary matching scores. We then feed the output into our affinity-aware decoder along with the projected feature maps for guiding the segmentation process. Combining these components, we conduct experiments to demonstrate the effectiveness of the proposed method, and our method sets a new state-of-the-art for all the standard benchmarks in few-shot segmentation task. Furthermore, we find that the proposed method attains state-of-the-art performance even for the standard benchmarks in semantic correspondence task although not specifically designed for this task. We also provide an extensive ablation study to validate our architectural choices. The trained weights and codes are available at: https: //seokju-cho.github.io/VAT/.

show abstract

Section: Introductionmentioning

confidence: 82%

Section: Related Workmentioning

confidence: 99%

Section: Problem Formulationmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Cost Aggregation Is All You Need for Few-Shot Segmentation

Hong¹,

Cho²,

Nam³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…The task of few-shot segmentation was first discussed in vision applications [7] and has since been adapted into the medical imaging field [8]. The commonly adopted approach is prototypical learning [9], where the foreground features of the support image is pooled into a prototype vector to represent the novel class.…”

Section: Related Workmentioning

confidence: 99%

Few-shot image segmentation for cross-institution male pelvic organs using registration-assisted prototypical learning

Li¹,

Fu²,

Yang³

et al. 2022

Preprint

View full text Add to dashboard Cite

The ability to adapt medical image segmentation networks for a novel class such as an unseen anatomical or pathological structure, when only a few labelled examples of this class are available from local healthcare providers, is sought-after. This potentially addresses two widely recognised limitations in deploying modern deep learning models to clinical practice, expertise-and-labour-intensive labelling and cross-institution generalisation. This work presents the first 3D few-shot interclass segmentation network for medical images, using a labelled multi-institution dataset from prostate cancer patients with eight regions of interest. We propose an image alignment module registering the predicted segmentation of both query and support data, in a standard prototypical learning algorithm, to a reference atlas space. The built-in registration mechanism can effectively utilise the prior knowledge of consistent anatomy between subjects, regardless whether they are from the same institution or not. Experimental results demonstrated that the proposed registration-assisted prototypical learning significantly improved segmentation accuracy (p-values<0.01) on query data from a holdout institution, with varying availability of support data from multiple institutions. We also report the additional benefits of the proposed 3D networks with 75% fewer parameters and an arguably simpler implementation, compared with existing 2D few-shot approaches that segment 2D slices of volumetric medical images.

show abstract

Sustainable Chemistry at Sungkyunkwan University

Park

2015

ChemSusChem

View full text Add to dashboard Cite

Special Issue: Sustainable Chemistry at Sungkyunkwan University. Sustainable chemistry is key to the development of efficient renewable energies, which will become more and more important in order to combat global warming. In this Editorial, guest editor Prof. Nam‐Gyu Park describes the context of this Special Issue on top‐quality research towards sustainability performed at Sungkyunkwan University (SKKU) in Korea. Scientists at SKKU work on, for example, photovoltaic solar cells to generate low‐cost electricity, lithium batteries and capacitors to store electricity, piezoelectric nanogenerators, thermoelectric devices, hydrogen generation, and fuel cells.

show abstract

One-Shot Learning for Semantic Segmentation

Cited by 97 publications

References 0 publications

Cost Aggregation Is All You Need for Few-Shot Segmentation

Cost Aggregation Is All You Need for Few-Shot Segmentation

Few-shot image segmentation for cross-institution male pelvic organs using registration-assisted prototypical learning

Sustainable Chemistry at Sungkyunkwan University

Contact Info

Product

Resources

About