No abstract

Detailed analysis of secondary envelopment of the herpesvirus human cytomegalovirus (HCMV) by transmission electron microscopy (TEM) is crucial for understanding the formation of infectious virions. Here, we present a convolutional neural network (CNN) that automatically recognises cytoplasmic capsids and distinguishes between three HCMV capsid envelopment stages in TEM images. 315 TEM images containing 2,610 expert‐labelled capsids of the three classes were available for CNN training. To overcome the limitation of small training datasets and thus poor CNN performance, we used a deep learning method, the generative adversarial network (GAN), to automatically increase our labelled training dataset with 500 synthetic images and thus to 9,192 labelled capsids. The synthetic TEM images were added to the ground truth dataset to train the Faster R‐CNN deep learning‐based object detector. Training with 315 ground truth images yielded an average precision (AP) of 53.81% for detection, whereas the addition of 500 synthetic training images increased the AP to 76.48%. This shows that generation and additional use of synthetic labelled images for detector training is an inexpensive way to improve detector performance. This work combines the gold standard of secondary envelopment research with state‐of‐the‐art deep learning technology to speed up automatic image analysis even when large labelled training datasets are not available.

Semantic segmentation of electron microscopy images using deep learning methods is a valuable tool for the detailed analysis of organelles and cell structures. However, these methods require a large amount of labeled ground truth data that is often unavailable. To address this limitation, we present a weighted average ensemble model that can automatically segment biological structures in electron microscopy images when trained with only a small dataset. Thus, we exploit the fact that a combination of diverse base-learners is able to outperform one single segmentation model. Our experiments with seven different biological electron microscopy datasets demonstrate quantitative and qualitative improvements. We show that the Grad-CAM method can be used to interpret and verify the prediction of our model. Compared with a standard U-Net, the performance of our method is superior for all tested datasets. Furthermore, our model leverages a limited number of labeled training data to segment the electron microscopy images and therefore has a high potential for automated biological applications.

No abstract

Image processing algorithms in pathology commonly include automated decision points such as classifications. While this enables efficient automation, there is also a risk that errors are induced. A different paradigm is to use image processing for enhancements without introducing explicit classifications. Such enhancements can help pathologists to increase efficiency without sacrificing accuracy. In our work, this paradigm has been applied to Ki-67 hot spot detection. Ki-67 scoring is a routine analysis to quantify the proliferation rate of tumor cells. Cell counting in the hot spot, the region of highest concentration of positive tumor cells, is a method increasingly used in clinical routine. An obstacle for this method is that while hot spot selection is a task suitable for low magnification, high magnification is needed to discern positive nuclei, thus the pathologist must perform many zooming operations. We propose to address this issue by an image processing method that increases the visibility of the positive nuclei at low magnification levels. This tool displays the modified version at low magnification, while gradually blending into the original image at high magnification. The tool was evaluated in a feasibility study with four pathologists targeting routine clinical use. In a task to compare hot spot concentrations, the average accuracy was 75±4.1% using the tool and 69±4.6% without it (n=4). Feedback on the system, gathered from an observer study, indicate that the pathologists found the tool useful and fitting in their existing diagnostic process. The pathologists judged the tool to be feasible for implementation in clinical routine.