A deep learning-based segmentation pipeline for profiling cellular morphodynamics using multiple types of live cell microscopy

Abstract

MOTIVATION Quantitative studies of cellular morphodynamics rely on extracting leading-edge velocity time series based on accurate cell segmentation from live cell imaging. However, live cell imaging has numerous challenging issues regarding accurate edge localization. Fluorescence live cell imaging produces noisy and low-contrast images due to phototoxicity and photobleaching. While phase contrast microscopy is gentle to live cells, it suffers from the halo and shade-off artifacts that cannot be handled by conventional segmentation algorithms. Here, we present a deep learning-based pipeline, termed MARS-Net (Multiple-microscopy-type-based Accurate and Robust Segmentation Network), that utilizes transfer learning and data from multiple types of microscopy to localize cell edges with high accuracy, allowing quantitative profiling of cellular morphodynamics. SUMMARY To accurately segment cell edges and quantify cellular morphodynamics from live-cell imaging data, we developed a deep learning-based pipeline termed MARS-Net (multiple-microscopy-type-based accurate and robust segmentation network). MARS-Net utilizes transfer learning and data from multiple types of microscopy to localize cell edges with high accuracy. For effective training on distinct types of live-cell microscopy, MARS-Net comprises a pretrained VGG19 encoder with U-Net decoder and dropout layers. We trained MARS-Net on movies from phase-contrast, spinning-disk confocal, and total internal reflection fluorescence microscopes. MARS-Net produced more accurate edge localization than the neural network models trained with single-microscopy-type datasets. We expect that MARS-Net can accelerate the studies of cellular morphodynamics by providing accurate pixel-level segmentation of complex live-cell datasets.

Protocol for live cell image segmentation to profile cellular morphodynamics using MARS-Net

“…MARS-Net ( Jang et al., 2021 ) takes a transfer learning approach ( Bertasius et al, 2015 ; Iglovikov et al, 2018 ; Kim et al., 2018 ; Long et al, 2015 ; Vaidyanathan et al., 2021 ) by integrating ImageNet-pretrained VGG19 encoder and U-Net decoder with additional dropout layers ( Deng et al., 2009 ; Ronneberger et al., 2015 ; Simonyan and Zisserman, 2015 ; Srivastavanitish et al., 2014 ), trained on the datasets from multiple types of microscopy. MARS-Net accepts the segmented images generated in Part 3 of this protocol.…”

“…The visualization codes in this Part show the cross-validation results by averaging the evaluation results from the segmentation of each movie. For more details on what each plot describes, refer to the MARS-Net ( Jang et al., 2021 ). CRITICAL: Correspondence Algorithm( Arbelaez et al., 2010 ) necessary for calculating F1, precision and recall is only supported in Linux.…”

“…Here, we provide the protocol for installing MARS-Net, labeling images, training MARS-Net for edge localization, evaluating the trained models’ performance, and performing the quantitative profiling of cellular morphodynamics. For complete details on the use and execution of this protocol, please refer to Jang et al. (2021) .…”

“…For complete details on the use and execution of this protocol, please refer to Jang et al. (2021) .…”

Recent advances in microfluidics-based cell migration research

“…Jang et al developed a MARS-Net (an accurate and robust segmentation network on multiple microscopy type) to localize cell edges to study morphological dynamics in migrating cells. 118 MARS-Net contains a pre-trained VGG19 encoder with a U-Net decoder and dropout layers. This network was trained on time-lapse images using three different types of microscopic imaging modes: phase-contrast, spinning-disk confocal, and total internal reflection fluorescence microscopy.…”