Automated Classification of Apoptosis in Phase Contrast Microscopy Using Capsule Network

Mobiny, Aryan; Lu, Hengyang; Nguyen, Hien Van; Roysam, Badrinath; Varadarajan, Navin

doi:10.1109/tmi.2019.2918181

Cited by 55 publications

(49 citation statements)

References 27 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…They can serve as an additional source of correlation to resolve ambiguities caused by overlapping cells of abnormal cell shapes. Our recent work [29] supports this hypothesis by showing that the temporal information in time lapse videos can significantly improve the cell apoptosis classification accuracy. Finally, our pipeline can be extended to detect and segment other subcellular organelles like mitochondria.…”

Section: Discussionmentioning

confidence: 65%

“…Our paper demonstrates a critical advance in this domain by enabling the detection of not just cellular boundaries but also nuclear boundaries within individual cells. We have recently reported the ability to detect apoptosis within individual cells using just the phase image [29] and thus we are developing the toolkit of algorithms essential to be able to advance cellular immunotherapy. Broadly, we envision the experimental methods and the associated advances in the detection of subcellular organelles will enable the identification of T cells associated with clinical benefit in the context of adoptive cellular immunotherapy.…”

Section: Discussionmentioning

confidence: 99%

See 1 more Smart Citation

Phasetime: Deep Learning Approach to Detect Nuclei in Time Lapse Phase Images

Yuan

Rezvan

et al. 2019

JCM

Self Cite

View full text Add to dashboard Cite

Time lapse microscopy is essential for quantifying the dynamics of cells, subcellular organelles and biomolecules. Biologists use different fluorescent tags to label and track the subcellular structures and biomolecules within cells. However, not all of them are compatible with time lapse imaging, and the labeling itself can perturb the cells in undesirable ways. We hypothesized that phase image has the requisite information to identify and track nuclei within cells. By utilizing both traditional blob detection to generate binary mask labels from the stained channel images and the deep learning Mask RCNN model to train a detection and segmentation model, we managed to segment nuclei based only on phase images. The detection average precision is 0.82 when the IoU threshold is to be set 0.5. And the mean IoU for masks generated from phase images and ground truth masks from experts is 0.735. Without any ground truth mask labels during the training time, this is good enough to prove our hypothesis. This result enables the ability to detect nuclei without the need for exogenous labeling.

show abstract

Section: Discussionmentioning

confidence: 65%

Section: Discussionmentioning

confidence: 99%

Phasetime: Deep Learning Approach to Detect Nuclei in Time Lapse Phase Images

Yuan

Rezvan

et al. 2019

JCM

Self Cite

View full text Add to dashboard Cite

show abstract

“…In natural language processing, Zhang et al 14 using capsule network to extract relationship, Du et al 15 proposed a new hybrid neural network based on emotion classification capsules, and McIntosh et al 16 applied multimodal capsule routing to action video segmentation. In medicine, CapsNet has been used to predict Alzheimer disease 17 , automatically classify apoptosis 18 , identify sign language 19 , and classify brain tumor types 20 .…”

Section: Related Workmentioning

confidence: 99%

DA-CapsNet: dual attention mechanism capsule network

Huang

Zhou

2020

Sci Rep

View full text Add to dashboard Cite

A capsule network (CapsNet) is a recently proposed neural network model with a new structure. The purpose of CapsNet is to form activation capsules. In this paper, our team proposes a dual attention mechanism capsule network (DA-CapsNet). In DA-CapsNet, the first layer of the attention mechanism is added after the convolution layer and is referred to as Conv-Attention; the second layer is added after the PrimaryCaps and is referred to as Caps-Attention. The experimental results show that DA-CapsNet performs better than CapsNet. For MNIST, the trained DA-CapsNet is tested in the testset, the accuracy of the DA-CapsNet is 100% after 8 epochs, compared to 25 epochs for CapsNet. For SVHN, CIFAR10, FashionMNIST, smallNORB, and COIL-20, the highest accuracy of DA-CapsNet was 3.46%, 2.52%, 1.57%, 1.33% and 1.16% higher than that of CapsNet. And the results of image reconstruction in COIL-20 show that DA-CapsNet has a more competitive performance than CapsNet.

show abstract

“…Performance of the conventional CAD systems relied on the intermediate image processing stages such as extraction of hand-crafted features [27,28]. In recent years, deep learning-based approaches have attracted considerable interest in the computer vision and machine learning community including the medical imaging domain [29,30,31]. Convolutional neural networks (CNN) can automatically extract the higher-level representations directly from raw input images [32].…”

Section: Related Workmentioning

confidence: 99%

Risk-Aware Machine Learning Classifier for Skin Lesion Diagnosis

Mobiny

Singh

Nguyen

2019

JCM

Self Cite

View full text Add to dashboard Cite

Knowing when a machine learning system is not confident about its prediction is crucial in medical domains where safety is critical. Ideally, a machine learning algorithm should make a prediction only when it is highly certain about its competency, and refer the case to physicians otherwise. In this paper, we investigate how Bayesian deep learning can improve the performance of the machine–physician team in the skin lesion classification task. We used the publicly available HAM10000 dataset, which includes samples from seven common skin lesion categories: Melanoma (MEL), Melanocytic Nevi (NV), Basal Cell Carcinoma (BCC), Actinic Keratoses and Intraepithelial Carcinoma (AKIEC), Benign Keratosis (BKL), Dermatofibroma (DF), and Vascular (VASC) lesions. Our experimental results show that Bayesian deep networks can boost the diagnostic performance of the standard DenseNet-169 model from 81.35% to 83.59% without incurring additional parameters or heavy computation. More importantly, a hybrid physician–machine workflow reaches a classification accuracy of 90 % while only referring 35 % of the cases to physicians. The findings are expected to generalize to other medical diagnosis applications. We believe that the availability of risk-aware machine learning methods will enable a wider adoption of machine learning technology in clinical settings.

show abstract

Automated Classification of Apoptosis in Phase Contrast Microscopy Using Capsule Network

Cited by 55 publications

References 27 publications

Phasetime: Deep Learning Approach to Detect Nuclei in Time Lapse Phase Images

Phasetime: Deep Learning Approach to Detect Nuclei in Time Lapse Phase Images

DA-CapsNet: dual attention mechanism capsule network

Risk-Aware Machine Learning Classifier for Skin Lesion Diagnosis

Contact Info

Product

Resources

About