Robust Prototypical Few-Shot Organ Segmentation with Regularized Neural-ODEs

Abstract

Despite the tremendous progress made by deep learning models in image semantic segmentation, they typically require large annotated examples, and in-creasing attention is being diverted to problem settings like Few-Shot Learning (FSL) where only a small amount of annotation is needed for generalisation to novel classes. This is especially seen in medical domains where dense pixel-level annotations are expensive to obtain. In this pa-per, we propose Regularized Prototypical Neural Ordinary Differential Equation (R-PNODE), a method that leverages intrinsic properties of Neural-ODEs, assisted and enhanced by additional cluster and consistency losses to perform Few-Shot Segmentation (FSS) of organs. R-PNODE con-strains support and query features from the same classes to lie closer in the representation space thereby improving the performance over the existing Convolutional Neural Network (CNN) based FSS methods. We further demon-strate that while many existing Deep CNN based methods tend to be extremely vulnerable to adversarial attacks, R-PNODE exhibits increased adversarial robustness for a wide array of these attacks. We experiment with three pub-licly available multi-organ segmentation datasets in both in-domain and cross-domain FSS settings to demonstrate the efficacy of our method. In addition, we perform experiments with seven commonly used adversarial attacks in various settings to demonstrate R-PNODE's robustness. R-PNODE outperforms the baselines for FSS by significant margins and also shows superior performance for a wide array of attacks varying in intensity and design.

A Language-Guided Benchmark for Weakly Supervised Open Vocabulary Semantic Segmentation

“…Similar to how humans can relate visual understanding of classes with similar-in-meaning names or categories, ZSS methods generalize semantic visual information using the semantic textual information provided by language models. Another slightly relaxed data efficient setting is FSS [8,9,10,11,12,13,14], where the model is expected to generalize to unseen classes but is additionally given few support images with annotated unseen target classes. Typical FSS methods demonstrate admirable performance using support samples ranging from one to five examples for every unseen category.…”