Physics Discovery in Nanoplasmonic Systems via Autonomous Experiments in Scanning Transmission Electron Microscopy

Abstract

Physics-driven discovery in an autonomous experiment has emerged as a dream application of machine learning in physical sciences. Here, this work develops and experimentally implements a deep kernel learning (DKL) workflow combining the correlative prediction of the target functional response and its uncertainty from the structure, and physics-based selection of acquisition function, which autonomously guides the navigation of the image space. Compared to classical Bayesian optimization (BO) methods, this approach allows to capture the complex spatial features present in the images of realistic materials, and dynamically learn structure-property relationships. In combination with the flexible scalarizer function that allows to ascribe the degree of physical interest to predicted spectra, this enables physical discovery in automated experiment. Here, this approach is illustrated for nanoplasmonic studies of nanoparticles and experimentally implemented in a truly autonomous fashion for bulk-and edge plasmon discovery in MnPS 3 , a lesser-known beam-sensitive layered 2D material. This approach is universal, can be directly used as-is with any specimen, and is expected to be applicable to any probe-based microscopic techniques including other STEM modalities, scanning probe microscopies, chemical, and optical imaging.

Machine learning in electron microscopy for advanced nanocharacterization: current developments, available tools and future outlook

“…Meaningfully, K. Roccapriore et al could automate the dynamic STEM exploration and EELS acquisition, through training a deep kernel capable of actively distinguishing physically-significant regions. 39 However, acquisition and alignment automation are not limited to experiment optimisation, and was key to open unprecedented experimental setups impossible otherwise. In that sense, E. Rotunno et al trained a CNN to align an orbital angular momentum sorter in the context of beam shaping experiments.…”

Learning the right channel in multimodal imaging: automated experiment in piezoresponse force microscopy

“…However, in this case the process is slow and heavily biased by operator experience and expectations. An alternative approach is that of dense grid-based measurements, such as force-volume 26 , piezoresponse spectroscopy, piezoresponse nonlinearity measurements in SPM [27][28][29][30][31] , hyperspectral electron energy loss spectroscopy (EELS) measurements in scanning transmission electron microscopy (STEM) 32 , photoluminescence lifetime measurement in optical microscopy 19 , or electron diffraction measurement in electron microscopy 29 . However, the grid measurements tend to be time consuming and are often limited or impossible for circumstances where the probe or the sample degrade rapidly with measurements.…”

“…For example, we aim to discover which microstructural element has the best predictive capacity for the functional property encoded in polarization hysteresis loop or resonance frequency hysteresis loop such as maximal loop area, imprint bias, or more complex functionals of the loop shape. For unimodal imaging, this approach have recently been demonstrated for STEM-EELS, 4D STEM, and band excitation piezoresponse spectroscopy (BEPS) 27,32,37,38 . In these studies, we have discovered which features in image space are most predictive of the specific functionalities determined via spectral measurements, for example localization of the hysteresis loops with the maximal area at specific domain walls or emergence of low energy plasmons at the edges of 2D material flakes.…”

A dynamic Bayesian optimized active recommender system for curiosity-driven partially “Human-in-the-loop” automated experiments

“…The achievable progress in the field of automated and autonomous experiments, and the idea of 'self-driving' laboratories more generally, hinges on the ability of probabilistic machine learning models to be used to rapidly identify areas of the parameter space that have a high (modeled) likelihood of optimizing target properties of interest. [1][2][3][4][5] Recent examples include explorations of chemical space 6 in the synthesis of nanoparticles 7 and thin films for photovoltaic applications, 8,9 . Additionally, numerous examples exist of autonomous microscopes that can be used to identify structure-property relationships in both electron 4 and scanning probe spectroscopies, 10,11 as well as scattering measurements at the beamline, for e.g., efficient capture of diffraction patterns for phase mapping or for strain imaging.…”