Practical segmentation of nuclei in brightfield cell images with neural networks trained on fluorescently labelled samples

Abstract

Identifying nuclei is a standard first step when analysing cells in microscopy images. The traditional approach relies on signal from a DNA stain, or fluorescent transgene expression localised to the nucleus. However, imaging techniques that do not use fluorescence can also carry useful information. Here, we used brightfield and fluorescence images of fixed cells with fluorescently labelled DNA, and confirmed that three convolutional neural network architectures can be adapted to segment nuclei from the brightfield channel, relying on fluorescence signal to extract the ground truth for training. We found that U-Net achieved the best overall performance, Mask R-CNN provided an additional benefit of instance segmentation, and that DeepCell proved too slow for practical application. We trained the U-Net architecture on over 200 dataset variations, established that accurate segmentation is possible using as few as 16 training images, and that models trained on images from similar cell lines can extrapolate well. Acquiring data from multiple focal planes further helps distinguish nuclei in the samples. Overall, our work helps to liberate a fluorescence channel reserved for nuclear staining, thus providing more information from the specimen, and reducing reagents and time required for preparing imaging experiments.

ArtSeg: Rapid Artifact Segmentation and Removal in Brightfield Cell Microscopy Images

“…Overall, the datasets cover nine different cell lines, fixed and live cells, two different plate formats and two microscopes. The datasets provenances have been described previously 3,28,4,29 and we briefly describe their most important properties here.…”

“…Our weakly supervised artifact segmentation pipeline combines the ScoreCAM model 18 that highlights areas of the image most useful for differentiating between clean and artifact-containing images with U-Net model 4 that directly classifies pixels into categories. We call this pipeline “ScoreCAM-U-Net” (Figure 1).…”

“…Artifact segmentation pipeline - ScoreCAM-U-Net. During training (top), ScoreCAM 18 (purple) is used to generate pixel-level probability maps of artifacts and the corresponding binary masks that are used to train the U-Net 4 segmentation model (blue). During the inference (bottom), the trained U-Net (blue) is used to segment artifacts from the images that were deemed to contain artifacts (image with red borders) by the ScoreCAM (purple).…”

“…Among the different imaging modalities, brightfield illumination with transmitted light is the simplest to acquire while avoiding damaging the sample 1 . The usefulness of this technology has led to its widespread adoption 2–4 , and thereby to a dramatic increase in the volumes of microscopy data. However, the automated analysis techniques required to extract information at scale are often hindered by the artifacts present in the images 5,6 .…”

“…To our knowledge, this is the first attempt to segment artifacts in microscopy images in a weakly supervised way. We introduce ScoreCAM-U-Net, a model that combines the informative pixel-level 4 and cheap-to-generate image-level 18 annotation schemes, and accurately detects artifacts in held-out samples. As training is performed using only image-level labels, generating training data is orders of magnitude cheaper, but without substantially sacrificing performance compared to pixel-level data.…”