The advantages of sub-sampling and Inpainting for scanning transmission electron microscopy

Abstract

Images and spectra obtained from aberration corrected scanning transmission electron microscopes (STEM) are now used routinely to quantify the morphology, structure, composition, chemistry, bonding, and optical/electronic properties of nanostructures, interfaces, and defects in many materials/biological systems. However, obtaining quantitative and reproducible atomic resolution observations from some experiments is actually harder with these ground-breaking instrumental capabilities, as the increase in beam current from using the correctors brings with it the potential for electron beam modification of the specimen during image acquisition. This beam effect is even more acute for in situ STEM observations, where the desired outcome being investigated is a result of a series of complicated transients, all of which can be modified in unknown ways by the electron beam. The aim in developing and applying new methods in STEM is, therefore, to focus on more efficient use of the dose that is supplied to the sample and to extract the most information from each image (or set of images). For STEM (and for that matter, all electron/ion/photon scanning systems), one way to achieve this is by sub-sampling the image and using Inpainting algorithms to reconstruct it. By separating final image quality from overall dose in this way and manipulating the dose distribution to be best for the stability of the sample, images can be acquired both faster and with less beam effects. In this paper, the methodology behind sub-sampling and Inpainting is described, and the potential for Inpainting to be applied to novel real time dynamic experiments will be discussed.

Towards real‐time STEM simulations through targeted subsampling strategies

“…In practice, however, it is found that line‐hop sampling (i.e. random walk) 61,62 is a better alternative sampling strategy for experimental data and it balances sparsity of acquisition and the limiting effects of hysteresis. UDS sampling is the optimal set up for image recovery, 35 however is limited in experiment due to hysteresis 61 .…”

“…random walk) 61,62 is a better alternative sampling strategy for experimental data and it balances sparsity of acquisition and the limiting effects of hysteresis. UDS sampling is the optimal set up for image recovery, 35 however is limited in experiment due to hysteresis 61 . Comparison of UDS and line‐hop can be found in previous work 57 and readers are referred to Figure S6 for examples of different mask types.…”

“…UDS sampling is the optimal set up for image recovery, 35 however is limited in experiment due to hysteresis. 61 Comparison of UDS and line-hop can be found in previous work 57 and readers are referred to Figure S6 for examples of different mask types.…”

High‐throughput determination of grain size distributions by EBSD with low‐discrepancy sampling

“…Several works in the literature have demonstrated significant improvements in reconstructions from low‐density grid sampling by using machine learning algorithms for in‐painting or reconstruction 31–33 . Such methods are largely complementary to ours, as they focus on how to reconstruct the image from the given points rather than which points should be sampled.…”

“…Several works in the literature have demonstrated significant improvements in reconstructions from low-density grid sampling by using machine learning algorithms for inpainting or reconstruction. [31][32][33] Such methods are largely complementary to ours, as they focus on how to reconstruct the image from the given points rather than which points should be sampled. It is possible that combining these methods with efficient quasi-random sampling of the types evaluated in this work could lead to further improvements in the efficiency of EBSD characterization.…”