Particle Recognition on Transmission Electron Microscopy Images Using Computer Vision and Deep Learning for Catalytic Applications

Abstract

Recognition and measuring particles on microscopy images is an important part of many scientific studies, including catalytic investigations. In this paper, we present the results of the application of deep learning to the automated recognition of nanoparticles deposited on porous supports (heterogeneous catalysts) on images obtained by transmission electron microscopy (TEM). The Cascade Mask-RCNN neural network was used. During the training, two types of objects were labeled on raw TEM images of ‘real’ catalysts: visible particles and overlapping particle projections. The trained neural network recognized nanoparticles in the test dataset with 0.71 precision and 0.72 recall for both classes of objects and 0.84 precision and 0.79 recall for visible particles. The developed model is integrated into the open-access web service ‘ParticlesNN’, which can be used by any researcher in the world. Instead of hours, TEM data processing per one image analysis is reduced to a maximum of a couple of minutes and the divergence of mean particle size determination is approximately 2% compared to manual analysis. The proposed tool encourages accelerating catalytic research and improving the objectivity and accuracy of analysis.

FPGA and computer-vision-based atom tracking technology for scanning probe microscopy

“…Hardware implementation can also be applied to other SPM systems. [10,31] Furthermore, the software can be implemented in other microscopy fields, such as optical microscopy, electron microscopy, [32][33][34] and so on.…”

Improving scientific image processing accessibility through development of graphical user interfaces for scikit-image

“…The TEM images used to test the usability of the GUIs have been previously published. 19 These images were initially used as part of a study focused on using neural networks to determine the average size of particles. In this instance, the TEM images are manipulated via the View Components in order to elicit new insights.…”

DLgram cloud service for deep‐learning analysis of microscopy images

“…For example, in biology, Cellpose (Stringer et al, 2021) and DeepCell Kiosk (Bannon et al, 2021) services have been created for cell recognition. We used this classical approach in materials science to recognize nanoparticles in scanning tunneling microscopy (STM) and transmission electron microscopy (TEM) images and created the free ParticlesNN web service (Nartova et al, 2022; Okunev, 2023a; Okunev et al, 2020). However, despite the high efficiency of the services, their use is limited by the types of objects on which the neural networks were trained.…”