NucMM Dataset: 3D Neuronal Nuclei Instance Segmentation at Sub-Cubic Millimeter Scale

Lin, Zudi; Wei, Donglai; Petkova, Mariela D.; Wu, Yuelong; Ahmed, Zergham; K, Krishna Swaroop; Zou, Silin; Wendt, Nils; Boulanger-Weill, Jonathan; Wang, Xueying; Dhanyasi, Nagaraju; Arganda‐Carreras, Ignacio; Engert, Florian; Lichtman, Jeff W.; Pfister, Hanspeter

doi:10.1007/978-3-030-87193-2_16

Cited by 20 publications

(22 citation statements)

References 27 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…When calculating the segmentation losses, we detach the synthesized image to avoid the segmentation objectives affecting the image translation results. U3D-BCD [16] uses multiple training augmentations like random missing, blurry and noisy regions (Fig. 3a).…”

Section: Methodsmentioning

confidence: 99%

“…3D instance segmentation from microscopy images is challenging due to the dense distribution of objects and unavoidable physical limitations in imaging (e.g., data is frequently anisotropic). Recent learning-based approaches tackle these challenges by first optimizing CNN-based models to predict representations calculated from the instance masks, including object boundary [5,22,26], affinity map [25,13], star-convex distance [27], flow-field [24] and the combination of multiple representations [16]. Watershed transform [7,29] and graph partition [12] can then be applied to convert the predicted representations into instance masks.…”

Section: Related Workmentioning

confidence: 99%

“…Instance segmentation approaches for microscopy images [24,27,26,16] usually predict instance representations computed from the permutation-invariant labels and then apply a decoding algorithm to yield the masks. In this work, we follow U3D-BCD [16] that predicts the binary foreground mask (B), instance contour map (C), and signed distance transform (D) as three output channels using a 3D U-Net [4], which are decoded by a marker-controlled watershed (MW) algorithm. The B and C channels are optimized with the binary cross-entropy loss (BCE), while D is regressed with the mean squared error (MSE).…”

Section: Instance Segmentation Lossesmentioning

confidence: 99%

“…3D Instance segmentation of cell nuclei is an essential topic attracting both biomedical and computer vision researchers [21,1,24,27,16]. Supervised deep learning with in-domain annotations (e.g., U-Net [22,4]) has become the dominant methodology for common imaging modalities.…”

Section: Introductionmentioning

confidence: 99%

“…Besides optimizing the translation and supervised segmentation losses as previous work [17], we introduce structural consistency and segmentation-based adversarial losses to better leverage the unlabeled domain images, connecting ideas from semi-supervised segmentation. Moreover, to incorporate data augmentations shown to enhance segmentation performance [13,16], we enforce the cycle consistency [28] of the reconstructed images to the clean images instead of the augmented ones, which acts as regularization and enable the model to restore corrupted regions.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Instance Segmentation of Unlabeled Modalities via Cyclic Segmentation GAN

Leander¹,

Lin²,

Zhang³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Instance segmentation for unlabeled imaging modalities is a challenging but essential task as collecting expert annotation can be expensive and time-consuming. Existing works segment a new modality by either deploying a pre-trained model optimized on diverse training data or conducting domain translation and image segmentation as two independent steps. In this work, we propose a novel Cyclic Segmentation Generative Adversarial Network (CySGAN) that conducts image translation and instance segmentation jointly using a unified framework. Besides the CycleGAN losses for image translation and supervised losses for the annotated source domain, we introduce additional self-supervised and segmentation-based adversarial objectives to improve the model performance by leveraging unlabeled target domain images. We benchmark our approach on the task of 3D neuronal nuclei segmentation with annotated electron microscopy (EM) images and unlabeled expansion microscopy (ExM) data. Our CySGAN outperforms both pre-trained generalist models and the baselines that sequentially conduct image translation and segmentation. Our implementation and the newly collected, densely annotated ExM nuclei dataset, named NucExM, are available at https://connectomics-bazaar.github.io/proj/CySGAN/index.html.

show abstract

Section: Methodsmentioning

confidence: 99%