We review the growing use of machine learning in electron microscopy (EM) driven in part by the availability of fast detectors operating at kiloHertz frame rates leading to large data sets that cannot be processed using manually implemented algorithms. We summarize the various network architectures and error metrics that have been applied to a range of EM-related problems including denoising and inpainting. We then provide a review of the application of these in both physical and life sciences, highlighting how conventional networks and training data have been specifically modified for EM.

We present a trainable segmentation method implemented within the python package ParticleSpy. The method takes user labelled pixels, which are used to train a classifier and segment images of inorganic nanoparticles from transmission electron microscope images. This implementation is based on the trainable Waikato Environment for Knowledge Analysis (WEKA) segmentation, but is written in python, allowing a large degree of flexibility and meaning it can be easily expanded using other python packages. We find that trainable segmentation offers better accuracy than global or local thresholding methods and requires as few as 100 user-labelled pixels to produce an accurate segmentation. Trainable segmentation presents a balance of accuracy and training time between global/local thresholding and neural networks, when used on transmission electron microscope images of nanoparticles. We also quantitatively investigate the effectiveness of the components of trainable segmentation, its filter kernels and classifiers, in order to demonstrate the use cases for the different filter kernels in ParticleSpy and the most accurate classifiers for different data types. A set of filter kernels is identified that are effective in distinguishing particles from background but that retain dissimilar features. In terms of classifiers, we find that different classifiers perform optimally for different image contrast; specifically, a random forest classifier performs best for high-contrast ADF images, but that QDA and Gaussian Naïve Bayes classifiers perform better for low-contrast TEM images.

We describe nNPipe for the automated analysis of morphologically diverse catalyst materials. Automated imaging routines and direct-electron detectors have enabled the collection of large data stacks over a wide range of sample positions at high temporal resolution. Simultaneously, traditional image analysis approaches are slow and hence unsuitable for large data stacks and consequently, researchers have progressively turned towards machine learning and deep learning approaches. Previous studies often detail work on morphologically uniform material systems with clearly discernible features, limited workable image sizes and training data that may be biased due to manual labelling. The nNPipe data-processing method consists of two standalone convolutional neural networks that were exclusively trained on multislice image simulations and enables fast analysis of 2048 × 2048 pixel images. Inference performance compared between idealised and real industrial catalytic samples and insights derived from subsequent data analysis are placed into the context of an automated imaging scenario.

No abstract