Noise2Atom: unsupervised denoising for scanning transmission electron microscopy images

Abstract

We propose an effective deep learning model to denoise scanning transmission electron microscopy (STEM) image series, named Noise2Atom, to map images from a source domain $\mathcal {S}$ S to a target domain $\mathcal {C}$ C , where $\mathcal {S}$ S is for our noisy experimental dataset, and $\mathcal {C}$ C is for the desired clear atomic images. Noise2Atom uses two external networks to apply additional constraints from the domain knowledge. This model requires no signal prior, no noise model estimation, and no paired training images. The only assumption is that the inputs are acquired with identical experimental configurations. To evaluate the restoration performance of our model, as it is impossible to obtain ground truth for our experimental dataset, we propose consecutive structural similarity (CSS) for image quality assessment, based on the fact that the structures remain much the same as the previous frame(s) within small scan intervals. We demonstrate the superiority of our model by providing evaluation in terms of CSS and visual quality on different experimental datasets.

Ensemble Machine‐Learning‐Based Analysis for In Situ Electron Diffraction

“…For the spot detection in this method, a threshold of intensity is set for filtering out the background noise, which also omits some weak spots. The recent development on denoising may enhance the signal‐noise ratio of experimental data, [ 23 ] which can be adopted into our algorithm to improve this issue in the future. (2) During the detecting process, we sometimes have to distinguish two overlapping spots, especially when dealing with diffraction patterns from many grains.…”

Machine learning in electron microscopy for advanced nanocharacterization: current developments, available tools and future outlook

“…The successful proofs of concept made additional denoising alternatives arise, including a denoising–noising Generative Adversarial Network (GAN) for the active denoising of STEM data (Fig. 2 GAN) 22–24 or case-independent denoising models that successfully outperformed classical restoration filters for both TEM and STEM. 23,25–31 Interestingly J. Vincent et al studied the latent features learned by the DL model to unveil the nature of the trained denoising dependencies to shine light on what is typically left as a black box.…”

Bringing into play automated electron microscopy data processing for understanding nanoparticulate electrocatalysts’ structure–property relationships

“…While registration-based methods are immensely useful when sequences of images are available, deep learning comes in handy when analyzing individual noisy images. Deep convolutional neural networks proved useful for restoring low-dose images of metal clusters on lighter support films [24] and an encoder-decoder-type deep learning model was developed for noise reduction and atomic column localization of different crystal structures, shown in Figure 2a [17].…”