Self-Support Few-Shot Semantic Segmentation

Abstract

Existing few-shot segmentation methods have achieved great progress based on the support-query matching framework. But they still heavily suffer from the limited coverage of intra-class variations from the few-shot supports provided. Motivated by the simple Gestalt principle that pixels belonging to the same object are more similar than those to different objects of same class, we propose a novel self-support matching strategy to alleviate this problem, which uses query prototypes to match query features, where the query prototypes are collected from high-confidence query predictions. This strategy can effectively capture the consistent underlying characteristics of the query objects, and thus fittingly match query features. We also propose an adaptive self-support background prototype generation module and self-support loss to further facilitate the self-support matching procedure. Our self-support network substantially improves the prototype quality, benefits more improvement from stronger backbones and more supports, and achieves SOTA on multiple datasets. Codes are at https://github.com/fanq15/SSP.

A Foreground Prototype-Based One-Shot Segmentation of Brain Tumors

“…The first module determines if the support and query image visually correspond, while the second dictates the network to focus on the targeted query objects. The method in [ 29 ] proposed a novel strategy that automated the process of matching query prototype and query image which are obtained by high-profile query image predictions. The tumor intensity, shape, and location may differ from image to image.…”

“…[ 32 ] proposed a one-shot image segmentation technique similar to ours. The method in [ 29 ] utilized a Gaussian mixture model to evaluate prototypical clusters of different classes. The method in [ 33 ] has implemented a few-shot-based U-Net architectures for detecting radiographic patterns and obtained an improvement in accuracy.…”