3D-surface reconstruction of cellular cryo-soft X-ray microscopy tomograms using semi-supervised deep learning

Abstract

Cryo-soft X-ray tomography (cryo-SXT) is a powerful method to investigate the ultrastructure of cells, offering resolution in the tens of nm range and strong contrast for membranous structures without requirement for labeling or chemical fixation. The short acquisition time and the relatively large volumes acquired allow for fast acquisition of large amounts of tomographic image data. Segmentation of these data into accessible features is a necessary step in gaining biologically relevant information from cryo-soft X-ray tomograms. However, manual image segmentation still requires several orders of magnitude more time than data acquisition. To address this challenge, we have here developed an end-to-end automated 3D-segmentation pipeline based on semi-supervised deep learning. Our approach is suitable for high-throughput analysis of large amounts of tomographic data, while being robust when faced with limited manual annotations and variations in the tomographic conditions. We validate our approach by extracting three-dimensional information on cellular ultrastructure and by quantifying nanoscopic morphological parameters of filopodia in mammalian cells.

Automated quantification of lipophagy in Saccharomyces cerevisiae from fluorescence and cryo-soft X-ray microscopy data using deep learning

“…CNNs have been used extensively for segmentation of computational tomography data in the clinical imaging communities [33,34,35,36]. However, few studies have used CNNs for segmentation of SXT data of individual cells [25,26,64]. Using an U-net-type architecture for the segmentation task and a traditional, watershed-based approach for the subsequent instance segmentation, we keep the computational burden and need for training data at a minimum and are nevertheless able to automatically segment vacuoles and LDs from X-ray image stacks and thereby to quantify the extent of droplet uptake into the vacuole.…”

“…Using an U-net-type architecture for the segmentation task and a traditional, watershed-based approach for the subsequent instance segmentation, we keep the computational burden and need for training data at a minimum and are nevertheless able to automatically segment vacuoles and LDs from X-ray image stacks and thereby to quantify the extent of droplet uptake into the vacuole. Our model thereby differs from previous attempts to segment organelles from X-ray tomogram reconstructions, as we use partially fluorescence annotated X-ray data for training [25, 26, 64].…”

“…SXT can be combined with cryo fluorescence microscopy, allowing for subcellular imaging and phenotype mapping with much higher throughput than possible in EM. A bottleneck in the analysis of 3D image stacks reconstructed from X-ray tomograms is the lack of suitable software tools for organelle segmentation, as only very few studies have attempted to use modern machine learning approaches to segment X-ray images in an automated manner for selected problems [25, 26]. This leaves the segmentation task often to tedious manual annotation of images in specialized settings and research environments.…”

3D surface reconstruction of cellular cryo-soft X-ray microscopy tomograms using semisupervised deep learning

“…The deep learning software developed for this study is publicly available via the GitHub repository under the MIT license https://github.com/noegroup/deep_sxt ( 74 ). The tomographic X-ray datasets are accessible under EMPIAR-11392 upon publication of this manuscript ( 75 ). The data used in training, evaluation and hyperparameter optimization procedure are accessible from the public repository http://dx.doi.org/10.17169/refubium-37222 ( 76 ).…”

Automated quantification of vacuole fusion and lipophagy in Saccharomyces cerevisiae from fluorescence and cryo-soft X-ray microscopy data using deep learning

“…SXT can be combined with cryo fluorescence microscopy, allowing for subcellular imaging and phenotype mapping with molecular specificity and over entire cell volumes. A bottleneck in the analysis of 3D image stacks reconstructed from X-ray tomograms is the lack of suitable software tools for organelle segmentation, as only very few studies have attempted to use modern machine learning approaches to segment X-ray images in an automated manner for selected problems [ 33 , 34 ]. This leaves the segmentation task often to tedious manual annotation of images in specialized settings and research environments.…”