Semantic Image Segmentation Using Scant Pixel Annotations

Abstract

The success of deep networks for the semantic segmentation of images is limited by the availability of annotated training data. The manual annotation of images for segmentation is a tedious and time-consuming task that often requires sophisticated users with significant domain expertise to create high-quality annotations over hundreds of images. In this paper, we propose the segmentation with scant pixel annotations (SSPA) approach to generate high-performing segmentation models using a scant set of expert annotated images. The models are generated by training them on images with automatically generated pseudo-labels along with a scant set of expert annotated images selected using an entropy-based algorithm. For each chosen image, experts are directed to assign labels to a particular group of pixels, while a set of replacement rules that leverage the patterns learned by the model is used to automatically assign labels to the remaining pixels. The SSPA approach integrates active learning and semi-supervised learning with pseudo-labels, where expert annotations are not essential but generated on demand. Extensive experiments on bio-medical and biofilm datasets show that the SSPA approach achieves state-of-the-art performance with less than 5% cumulative annotation of the pixels of the training data by the experts.

Super resolution-based methodology for self-supervised segmentation of microscopy images

“…The data were transformed into 7, 16-bit grayscale SEM images at a resolution of 1,024 × 758 px. More information regarding the data collection environments and measurements are discussed in the study by (Susarla et al, 2021 ; Abeyrathna et al, 2022a ; Chakravarthy et al, 2022 ). Sample images from each of the datasets and their corresponding GT masks are shown in Figure 1 .…”

“…Training with scarce annotated data leads to the overfitting problem (models that perform well on the data on which they are trained but not on others) (Hesamian et al, 2019 ), which makes the models unusable. Several approaches including data augmentation, dividing each image into multiple patches (Dou et al, 2017 ), and alternative ML approaches such as active learning (Chakravarthy et al, 2022 ) have been used to reduce the manual annotation effort. While many of these methods generally enhance the performance of deep learning models, not all techniques are suitable for analyzing microscopy image datasets that often consist of images of wide-ranging resolutions and magnifications.…”

On the Importance of Diversity When Training Deep Learning Segmentation Models with Error-Prone Pseudo-Labels

“…Secondly, the simple model such as random trees has poor generalization ability. Semi-supervised segmentation approaches usually augment manually labeled training data by generating pseudo-labels for the unlabeled data and using these to generate segmentation models [25]. Consistency regularization and entropy minimization represent two prevalent strategies in using unlabeled data [26].…”

An AI-based approach for detecting cells and microbial byproducts in low volume scanning electron microscope images of biofilms

“…These approaches are primarily based on the supervised machine learning methods which generate models that can automatically detect objects in images by training them on images annotated by experts. Due to the large volume of expert annotated images needed to train these models several alternative machine learning methods that work with scant annotated data are being developed (Tajbakhsh et al, 2020 ; Chakravarthy et al, 2022 ), Self-supervised learning is a popular method categorized under the techniques supported by scant annotations. In self-supervised learning, first representations are learned from input data without any expert annotations and then these learned representations are fine-tuned to perform the downstream object classification task using scant expert annotations.…”