Autonomous Single-Molecule Manipulation Based on Reinforcement Learning

Abstract

Building nanostructures one-by-one requires precise control of single molecules over many manipulation steps. The ideal scenario for machine learning algorithms is complex, repetitive, and time-consuming. Here, we show a reinforcement learning algorithm that learns how to control a single dipolar molecule in the electric field of a scanning tunneling microscope. Using about 2250 iterations to train, the algorithm learned to manipulate the molecule toward specific positions on the surface. Simultaneously, it generates physical insights into the movement as well as orientation of the molecule, based on the position where the electric field is applied relative to the molecule. This reveals that molecular movement is strongly inhibited in some directions, and the torque is not symmetric around the dipole moment.

Concept for the Real-Time Monitoring of Molecular Configurations during Manipulation with a Scanning Probe Microscope

“…An aspect that we have only briefly touched upon is the automation of molecular manipulation in a robotic manner. ,, The concept of a partially observable Markov decision process (POMDP) includes autonomous decision-making in which the policy for selecting an action is based on the agent’s belief regarding the configuration of the molecule, which can be obtained from the particle filter. In the future, configuration monitoring as demonstrated here and decision-making can thus be integrated seamlessly, such that a learning agent takes over the role of the experimenter who, in turn, sets up the rewards which control the agent’s behavior, thereby steering it toward a desired target (manipulation goal).…”

Machine learning and analytical methods for single-molecule conductance measurements

“…In related fields, there have been efforts to efficiently search for optimal experimental conditions through the utilisation of reinforcement learning. 171,172 These applications have the potential to aid in the discovery of appropriate experimental conditions for single-molecule measurements and facilitate the generation of more reliable data.…”

Synergizing human expertise and AI efficiency with language model for microscopy operation and automated experiment design *

“…Traditionally, mastering microscopy techniques for scientific discovery often requires years of training. Recently, rapid progress in artificial intelligence (AI) has assisted the development of AE in microscopes for physics discoveries [23][24][25][26][27][28]. AE-microscopes utilize a range of machine learning (ML) methods, including supervised ML [24,29], active learning [27,[30][31][32][33][34], and reinforcement learning [26,28,35,36] for on-the-fly data analysis and decision-making, automating microscope operation, and facilitating physics discovery.…”

“…Recently, rapid progress in artificial intelligence (AI) has assisted the development of AE in microscopes for physics discoveries [23][24][25][26][27][28]. AE-microscopes utilize a range of machine learning (ML) methods, including supervised ML [24,29], active learning [27,[30][31][32][33][34], and reinforcement learning [26,28,35,36] for on-the-fly data analysis and decision-making, automating microscope operation, and facilitating physics discovery. For example, convolutional neural networks (CNN) can be used to transform streaming microscope images into segmented images highlighting objects of interest, (e.g.…”