An Automated Scanning Transmission Electron Microscope Guided by Sparse Data Analytics

Abstract

Artificial intelligence (AI) promises to reshape scientific inquiry and enable breakthrough discoveries in areas such as energy storage, quantum computing, and biomedicine. Scanning transmission electron microscopy (STEM), a cornerstone of the study of chemical and materials systems, stands to benefit greatly from AI-driven automation. However, present barriers to low-level instrument control, as well as generalizable and interpretable feature detection, make truly automated microscopy impractical. Here, we discuss the design of a closed-loop instrument control platform guided by emerging sparse data analytics. We hypothesize that a centralized controller, informed by machine learning combining limited a priori knowledge and task-based discrimination, could drive on-the-fly experimental decision-making. This platform may unlock practical, automated analysis of a variety of material features, enabling new high-throughput and statistical studies.

Exploring Physics of Ferroelectric Domain Walls in Real Time: Deep Learning Enabled Scanning Probe Microscopy

“…The ferroelastic walls can be determined on the image directly via human eye; however, finding these automatically is a challenge. Over the last 5 years, deep convolutional neural networks (DCNN) have been broadly adopted in electron [ 50,51 ] and scanning probe microscopies. [ 52–54 ] However, while these techniques have amply demonstrated their potential for post‐acquisition data analysis, their implementation as a part of real time experiment is highly non‐trivial.…”

Forecasting of In Situ Electron Energy Loss Spectroscopy

“…For implementation in emerging AI systems, we envision the EELSTM forecasting model should be running continuously on a rolling buffer of EELS data and implemented in modelpredictive control frameworks for closed-loop feedback. 18 We expect that this approach will find powerful use in studies of high-speed phase transitions for fundamental studies of crystal nucleation and growth, battery cycling, mechanical deformation, and quantum behavior.…”

Machine learning in electron microscopy for advanced nanocharacterization: current developments, available tools and future outlook

“…Importantly, the identification of the features defining the score metric was mostly automated by the reviewed examples. [35][36][37][38][40][41][42][43][44]473 Then, the goal would be to reduce this sparsity or make it less meaningful. One option could be linking each tuneable parameter with a score modification, which would imply a hard encoding of the score.…”