Sparse Scanning Electron Microscopy Data Acquisition and Deep Neural Networks for Automated Segmentation in Connectomics

Abstract

With the growing importance of three-dimensional and very large field of view imaging, acquisition time becomes a serious bottleneck. Additionally, dose reduction is of importance when imaging material like biological tissue that is sensitive to electron radiation. Random sparse scanning can be used in the combination with image reconstruction techniques to reduce the acquisition time or electron dose in scanning electron microscopy. In this study, we demonstrate a workflow that includes data acquisition on a scanning electron microscope, followed by a sparse image reconstruction based on compressive sensing or alternatively using neural networks. Neuron structures are automatically segmented from the reconstructed images using deep learning techniques. We show that the average dwell time per pixel can be reduced by a factor of 2–3, thereby providing a real-life confirmation of previous results on simulated data in one of the key segmentation applications in connectomics and thus demonstrating the feasibility and benefit of random sparse scanning techniques for a specific real-world scenario.

Microstructure segmentation with deep learning encoders pre-trained on a large microscopy dataset

“…Automatic segmentation using classic computer vision techniques such as image thresholding and morphology operations 3,4 is much faster and repeatable, but difficult to implement and often not robust to slight changes in imaging or sample conditions. Recently, convolutional neural networks (CNN) pre-trained on ImageNet 5 have produced superior microscopy classification and segmentation results and are much easier to implement [6][7][8][9][10][11][12][13][14][15][16] . However, segmentation CNNs require expensively labeled training data to operate well and ImageNet pre-training does not adequately alleviate this problem when transferred to microscopy segmentation tasks because many of the learned filters are not applicable (e.g., those adapted to detect dogs).…”

Performance enhancement of a scanning electron microscope using a deep convolutional neural network

“…The increasing demand to perform metrology at nanoscales in various branches of science and engineering has driven efforts to enhance the performance of the scanning electron microscope (SEM). Performance enhancement mainly involves increasing the throughput of the SEM [1][2][3][4] without a degradation in the output image. The main factors that determine the SEM throughput are image resolution and frame rate.…”

“…This approach is followed in most of the machine learning-based techniques [22,23]. In recent years, the deep convolutional neural network (D-CNN) [24] has been used for image super-resolution [25] with application to microscopy [4,26,27]. D-CNN has also been used for noise reduction in low-dose CT [28], x-ray imaging [29], and MRI [30].…”

“…Though there are some similarities between the noise processes in SEMs and low-light imaging, low-dose computed (computer) tomographic imaging, etc it may be noted that noise processes in SEM imaging comes from several different stages due to the fact that the electron has charge and mass and its interaction with the specimen is quite different than photons [12]. In the context of SEMs, D-CNN has been used to reduce noise in line edge roughness measurements [7], random sparse scanning in electron microscopy [4], robust autofocusing deep learning network [40,41] and other work [42].…”

Learning Guided Electron Microscopy with Active Acquisition

“…Because of fast beam re-positioning, an arbitrary distribution of pixel locations has negligible impact on scanning time. Therefore, for an image where only 5% of pixels need scanning, one can ideally speed up its acquisition by 20 folds [1,23].…”