Cells respond heterogeneously to molecular and environmental perturbations. Phenotypic heterogeneity, wherein multiple phenotypes coexist in the same conditions, presents challenges when interpreting the observed heterogeneity. Advances in live cell microscopy allow researchers to acquire an unprecedented amount of live cell image data at high spatiotemporal resolutions. Phenotyping cellular dynamics, however, is a nontrivial task and requires machine learning (ML) approaches to discern phenotypic heterogeneity from live cell images. In recent years, ML has proven instrumental in biomedical research, allowing scientists to implement sophisticated computation in which computers learn and effectively perform specific analyses with minimal human instruction or intervention. In this review, we discuss how ML has been recently employed in the study of cell motility and morphodynamics to identify phenotypes from computer vision analysis. We focus on new approaches to extract and learn meaningful spatiotemporal features from complex live cell images for cellular and subcellular phenotyping.

Multidimensional scaling (MDS) is to recover a set of points by making use of noised pairwise Euclidean distances. In some situations, the observed Euclidean distances may contain large errors or even missing values. In such cases, the order of the distances is far more important than their magnitude. Non-metric multidimensional scaling (NMDS) is then to deal with this problem by taking use of the ordinal information. The challenge of NMDS is to tackle the large number of ordinal constraints on distances (for [Formula: see text] points, this will be of [Formula: see text]), which will slow down existing numerical algorithms. In this paper, we propose an ordinal weighted Euclidean distance matrix model for NMDS. By designing an ordinal weighted matrix, we get rid of the large number of ordinal constraints and tackle the ordinal constraints in a soft way. We then apply our model to image ranking. The key insight is to view the image ranking problem as NMDS in the kernel space. We conduct extensive numerical test on two state-of-the-art datasets: FG-NET aging dataset and MSRA-MM dataset. The results show the improvement of the proposed approach over the existing methods.

No abstract

Lens-free digital in-line holography (LDIH) produces cellular diffraction patterns from a large field of view that lens-based microscopes cannot achieve. It is a promising diagnostic tool allowing comprehensive cellular analysis with high-throughput capability. Since these diffraction images are far more complicated to discern, conventionally computational algorithms are used to reconstruct cellular images for the diffraction patterns. However, it is inefficient and prone to errors. Here, we developed a deep learning architecture, HoloNet to directly analyze the diffraction patterns from LDIH. In addition to the standard CNN (Convolutional Neural Network), the HoloNet includes a holo-branch that extracts large features from holograms and integrate them with the small features from the standard CNN. Using HoloNet, we accurately predicted the intensity values of ER/PR and HER2 in breast cancer cells and classified four known subtypes of breast cancer cells. Moreover, we applied a HoloNet dual embedding, which learns high-level diffraction features related to breast cancer cell types and the intensities of ER/PR and HER2. This allowed us to identify potential subtypes of breast cancer cells from the cell lines and the fine needle aspiration samples from breast cancer patients. We demonstrate that our HoloNet can enable LDIH to perform more detailed and rapid analyses of subtypes of breast cancer cells. Citation Format: Tzu-Hsi Song, Mengzhi Cao, Jouha Min, Hyungsoon Im, Hakho Lee, Kwonmoo Lee. Deep learning-based analysis of heterogeneity of breast cancer cells using lens-free digital in-line holography [abstract]. In: Proceedings of the AACR Virtual Special Conference on Artificial Intelligence, Diagnosis, and Imaging; 2021 Jan 13-14. Philadelphia (PA): AACR; Clin Cancer Res 2021;27(5_Suppl):Abstract nr PO-080.