Structure prediction of surface reconstructions by deep reinforcement learning

Abstract

We demonstrate how image recognition and reinforcement learning combined may be used to determine the atomistic structure of reconstructed crystalline surfaces. A deep neural network represents a reinforcement learning agent that obtains training rewards by interacting with an environment. The environment contains a quantum mechanical potential energy evaluator in the form of a density functional theory program. The agent handles the 3D atomistic structure as a series of stacked 2D images and outputs the next atom type to place and the atomic site to occupy. Agents are seen to require 1000-10 000 single point density functional theory evaluations, to learn by themselves how to build the optimal surface reconstructions of anatase TiO 2 (001)-(1 × 4) and rutile SnO 2 (110)-(4 × 1).

Global optimization of chemical cluster structures: Methods, applications, and challenges

“…Some sophisticated ML models have been used to guide generation of new structures. Reinforcement learning was applied in the reconstruction of material structures [283, 284]. In this scheme, a policy is defined as a subroutine that determines where a new atom is added based on the current state (atoms already constructed).…”

Perspective on integrating machine learning into computational chemistry and materials science

“…Beyond the significance to the discovery of new drugs and materials, finding stable equilibrium geometries and accessible transition states is a crucial element of computational molecular and materials discovery that typically involves tailored workflows. 168 As shown in Fig. 4, optimization problems in atomistic simulation span different scales from searching stable molecules across chemical space to charting the global energy landscape spanned by the chemical coordinates of a given molecule down to local structure relaxation and transition state search.…”

“…The approach is applicable to molecules as well as materials and has been showcased on graphene formation, and oxide surface reconstructions. 188 In the case of graphene, the method is able to generate graphene as the most stable two-dimensional phase starting from initially random atom placement. Bayesian optimisation has become a common tool to achieve efficient structure prediction for crystals, 189,190 surface reconstructions, 191 and hybrid organic/inorganic interfaces to name just a few examples.…”

Generating stable molecules using imitation and reinforcement learning

“…To improve the Q-value function we parameterize it by a neural network and update it as the agent collects new experience. Specifically, we evaluate the Q-values on a voxelated grid [35], which allows us to update the Q-value towards the highest final reward observed by the agent for a specific state-action pair (figure 1), which for a deterministic problem puts a lower bound on the optimal Q-value [44]. Additionally, this discretization allows for easy inference of the optimal atom placement by simply finding the voxel which maximizes the Q-value.…”

“…A notable limitation is a large database and no feedback-loop to improve the generated molecules beyond what is learned from the database. To remedy this, reinforcement learning (RL) methods have started to become a competitive alternative [27][28][29][30][31][32][33][34][35][36][37][38] to methods relying on existing databases. RL involves a model that produces molecules and obtains properties for these molecules, by some external means other than a database.…”