Extreme Points Derived Confidence Map as a Cue for Class-Agnostic Interactive Segmentation Using Deep Neural Network

Khan, Shadab; Shahin, Ahmed H.; Villafruela, Javier; Shen, Jianbing; Shao, Ling

doi:10.1007/978-3-030-32245-8_8

Cited by 16 publications

(8 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In Sakinis et al [23], the authors use the clicks as Gaussian kernels and put them in a separate input channel to an FCN to model user interactions via seed-point placing. Khan et al [24] extends the Gaussian kernel idea to a confidence map derived from extreme points that quantitatively encodes some priors. Majumder and Yao [25] transforms the positive and negative clicks into images based on superpixel and object proposals, so that image information can be used with clicks to generate a guidance map.…”

Section: Interactive Segmentationmentioning

confidence: 99%

Going to Extremes: Weakly Supervised Medical Image Segmentation

Roth

Yang

et al. 2021

MAKE

View full text Add to dashboard Cite

Medical image annotation is a major hurdle for developing precise and robust machine-learning models. Annotation is expensive, time-consuming, and often requires expert knowledge, particularly in the medical field. Here, we suggest using minimal user interaction in the form of extreme point clicks to train a segmentation model which, in effect, can be used to speed up medical image annotation. An initial segmentation is generated based on the extreme points using the random walker algorithm. This initial segmentation is then used as a noisy supervision signal to train a fully convolutional network that can segment the organ of interest, based on the provided user clicks. Through experimentation on several medical imaging datasets, we show that the predictions of the network can be refined using several rounds of training with the prediction from the same weakly annotated data. Further improvements are shown using the clicked points within a custom-designed loss and attention mechanism. Our approach has the potential to speed up the process of generating new training datasets for the development of new machine-learning and deep-learning-based models for, but not exclusively, medical image analysis.

show abstract

Section: Interactive Segmentationmentioning

confidence: 99%

Going to Extremes: Weakly Supervised Medical Image Segmentation

Roth

Yang

et al. 2021

MAKE

View full text Add to dashboard Cite

show abstract

“…Interactive Segmentation Interactive segmentation allow user to make edits by clicks (Xu et al 2016), scribbles (Grady et al 2005), bounding boxes (Rajchl et al 2016;Castrejon et al 2017), or extreme points (Maninis et al 2018;Khan et al 2019). Interactive segmentation models need to be pre-trained using a hold-out labeled dataset to make initial predictions before user interactions.…”

Section: Related Workmentioning

confidence: 99%

A Continual Learning Framework for Uncertainty-Aware Interactive Image Segmentation

Zheng

et al. 2021

AAAI

View full text Add to dashboard Cite

Deep learning models have achieved state-of-the-art performance in semantic image segmentation, but the results provided by fully automatic algorithms are not always guaranteed satisfactory to users. Interactive segmentation offers a solution by accepting user annotations on selective areas of the images to refine the segmentation results. However, most existing models only focus on correcting the current image's misclassified pixels, with no knowledge carried over to other images. In this work, we formulate interactive image segmentation as a continual learning problem and propose a framework to effectively learn from user annotations, aiming to improve the segmentation on both the current image and unseen images in future tasks while avoiding deteriorated performance on previously-seen images. It employs a probabilistic mask to control the neural network's kernel activation and extract the most suitable features for segmenting images in each task. We also apply a task-aware embedding to automatically infer the optimal kernel activation for initial segmentation and subsequent refinement. Interactions with users are guided through multi-source uncertainty estimation so that users can focus on the most important areas to minimize the overall manual annotation effort. Experiments are performed on both medical and natural image datasets to illustrate the proposed framework's effectiveness on basic segmentation performance, forward knowledge transfer, and backward knowledge transfer.

show abstract

“…In Sakinis et al (2019), the authors utilize the clicks as Gaussian kernels and put them in a separate input channel to an FCN to model user interactions via seed-point placing. Khan et al (2019) extends the Gaussian kernel idea to a confidence map derived from extreme points that quantitatively encodes some priors. Majumder and Yao (2019) transforms the positive and negative clicks into images based on superpixel and object proposals, so that image information can be utilized with clicks to generate a guidance map.…”

Section: Interactive Segmentationmentioning

confidence: 99%

Going to Extremes: Weakly Supervised Medical Image Segmentation

Roth¹,

Yang²,

Xu³

et al. 2020

Preprint

View full text Add to dashboard Cite

Medical image annotation is a major hurdle for developing precise and robust machine learning models. Annotation is expensive, time-consuming, and often requires expert knowledge, particularly in the medical field. Here, we suggest using minimal user interaction in the form of extreme point clicks to train a segmentation model which, in effect, can be used to speed up medical image annotation. An initial segmentation is generated based on the extreme points utilizing the random walker algorithm. This initial segmentation is then used as a noisy supervision signal to train a fully convolutional network that can segment the organ of interest, based on the provided user clicks. Through experimentation on several medical imaging datasets, we show that the predictions of the network can be refined using several rounds of training with the prediction from the same weakly annotated data. Further improvements are shown utilizing the clicked points within a custom-designed loss and attention mechanism. Our approach has the potential to speed up the process of generating new training datasets for the development of new machine learning and deep learning-based models for, but not exclusively, medical image analysis.

show abstract

Extreme Points Derived Confidence Map as a Cue for Class-Agnostic Interactive Segmentation Using Deep Neural Network

Cited by 16 publications

References 11 publications

Going to Extremes: Weakly Supervised Medical Image Segmentation

Going to Extremes: Weakly Supervised Medical Image Segmentation

A Continual Learning Framework for Uncertainty-Aware Interactive Image Segmentation

Going to Extremes: Weakly Supervised Medical Image Segmentation

Contact Info

Product

Resources

About