Adaptive Partial Scanning Transmission Electron Microscopy with Reinforcement Learning

Ede, Jeffrey M.

doi:10.48550/arxiv.2004.02786

Cited by 3 publications

(3 citation statements)

References 75 publications

(116 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…RL is applicable to optimization in the field of electron microscopy, such as adaptive scanning methods. For example, Ede [228] integrated an RNN with RL to optimize the scanning problem encountered when using STEM. In this case, RL based on an MDP is used to train the RNN to work together with a feed-forward CNN that accomplishes sparse scans.…”

Section: Application Of Rlmentioning

confidence: 99%

Advancing electron microscopy using deep learning

Chen,

Barnard

2024

J. Phys. Mater.

View full text Add to dashboard Cite

Electron microscopy, a sub-field of microanalysis, is a victim of its own success. The widespread use of electron microscopy for imaging molecules and materials has had an enormous impact on our understanding of countless systems and has accelerated impacts in drug discovery and materials design, for electronic, energy, environment and health applications. With this success a bottleneck has emerged, as the rate at which we can collect data has significantly exceeded the rate at which we can analyse it. Fortunately, this has coincided with the rise of advanced computational methods, including data science and machine learning. Deep learning, a sub-field of machine learning capable of learning from large quantities of data such as images, is ideally suited to overcome some of the challenges of electron microscopy at scale. There are a variety of different deep learning approaches relevant to the field, with unique advantages and disadvantages. In this review, we describe some well-established methods, with some recent examples, and introduce some new methods currently emerging in computer science. Our summary of deep learning is designed to guide electron microscopists to choose the right deep learning algorithm for their research and prepare for their digital future.

show abstract

Section: Application Of Rlmentioning

confidence: 99%

Advancing electron microscopy using deep learning

Chen,

Barnard

2024

J. Phys. Mater.

View full text Add to dashboard Cite

show abstract

“…Therefore, in this paper we use RL to design a scanning policy that acts optimally on each individual subject. In Scanning Transmission Electron Microscopy (STEM), a recent work by [50] proposes to use RL to guide the movement of the detector and uses a generator to generate reconstructed images. However, since the image modality is drastically different from CT, the proposed MDP (especially the state, action and architecture of the policy network) is vastly different from what is proposed in this paper.…”

Section: Related Workmentioning

confidence: 99%

Learning to Scan: A Deep Reinforcement Learning Approach for Personalized Scanning in CT Imaging

Shen¹,

Wang²,

Wu³

et al. 2020

Preprint

View full text Add to dashboard Cite

Computed Tomography (CT) takes X-ray measurements on the subjects to reconstruct tomographic images. As X-ray is radioactive, it is desirable to control the total amount of dose of X-ray for safety concerns. Therefore, we can only select a limited number of measurement angles and assign each of them limited amount of dose. Traditional methods such as compressed sensing usually randomly select the angles and equally distribute the allowed dose on them. In most CT reconstruction models, the emphasize is on designing effective image representations, while much less emphasize is on improving the scanning strategy. The simple scanning strategy of random angle selection and equal dose distribution performs well in general, but they may not be ideal for each individual subject. It is more desirable to design a personalized scanning strategy for each subject to obtain better reconstruction result. In this paper, we propose to use Reinforcement Learning (RL) to learn a personalized scanning policy to select the angles and the dose at each chosen angle for each individual subject. We first formulate the CT scanning process as an MDP, and then use modern deep RL methods to solve it. The learned personalized scanning strategy not only leads to better reconstruction results, but also shows strong generalization to be combined with different reconstruction algorithms. * Equal Contribution Preprint. Under review.

show abstract

“…34 LSTM models have been extensively applied to serial data, such as text, audio, and video, 35 and to materials science problems such as prediction of switching in ferroelectrics. 36 Despite their prevalence, there has been surprisingly little work on the use of LSTMs in electron microscopy, with limited examples including control of scan generation 37 and segmentation of biological images. 38 Given that in situ STEM data is acquired in serial fashion, we aim to evaluate the performance of LSTM for microscope data, with an eye toward practical implementation.…”

mentioning

confidence: 99%

Forecasting of In Situ Electron Energy Loss Spectroscopy

Spurgeon

Lewis

et al. 2022

Preprint

View full text Add to dashboard Cite

Forecasting models are a central part of many control systems, where high consequence decisions must be made on long latency control variables. These models are particularly relevant for emerging artificial intelligence (AI)-guided instrumentation, in which prescriptive knowledge is needed to guide autonomous decision-making. Here we describe the implementation of a long short-term memory model (LSTM) for forecasting of electron energy loss spectroscopy (EELS) data, one of the richest analytical probes of materials and chemical systems. We describe key considerations for data collection, preprocessing, training, validation, and benchmarking, showing how this approach can yield powerful predictive insight into order-disorder phase transitions. Finally, we comment on how such a model may integrate with emerging AI-guided instrumentation for powerful high-speed experimentation.

show abstract

Adaptive Partial Scanning Transmission Electron Microscopy with Reinforcement Learning

Cited by 3 publications

References 75 publications

Advancing electron microscopy using deep learning

Advancing electron microscopy using deep learning

Learning to Scan: A Deep Reinforcement Learning Approach for Personalized Scanning in CT Imaging

Forecasting of In Situ Electron Energy Loss Spectroscopy

Contact Info

Product

Resources

About