Solving the $k$-Sparse Eigenvalue Problem with Reinforcement Learning

Abstract

We examine the possibility of using a reinforcement learning (RL) algorithm to solve large-scale eigenvalue problems in which the desired the eigenvector can be approximated by a sparse vector with at most k nonzero elements, where k is relatively small compare to the dimension of the matrix to be partially diagonalized. This type of problem arises in applications in which the desired eigenvector exhibits localization properties and in large-scale eigenvalue computations in which the amount of computational resource is limited. When the positions of these nonzero elements can be determined, we can obtain the k-sparse approximation to the original problem by computing the eigenvalue of a k×k submatrix extracted from k rows and columns of the original matrix. We review a previously developed greedy algorithm for incrementally probing the positions of the nonzero elements in a k-sparse approximate eigenvector and show that the greedy algorithm can be improved by using an RL method to refine the selection of k rows and columns of the original matrix. We describe how to represent states, actions, rewards and policies in an RL algorithm designed to solve the k-sparse eigenvalue problem and demonstrate the effectiveness of the RL algorithm on two examples originating from quantum many-body physics.

Reinforcement Learning Configuration Interaction

“…In reinforcement learning, an agent is trained to take a series of actions in order to maximize a reward . Herein, we follow much of the notation and framework laid out for the k -sparse eigenproblem in ref and apply it to the sCI case. The reinforcement learning process proceeds in a series of episodes, where the agent explores the environment and obtains rewards or penalties for its actions.…”

“…Note that if an action is selected, the loop over S 1 is immediately terminated. Action pairs a = ( p , q ), p ∈ s , q ∉ s from the two lists are iterated over, and an action a is taken if it satisfies a Metropolis-like criterion as detailed in ref . If no action is selected, the learning procedure terminates. Once the action a is selected, the search policy terminates, the local reward r is computed according to eq , and the weights w and v are updated according to eqs and , respectively.…”

“…Note that if an action is selected, the loop over S 1 is immediately terminated. Action pairs a = ( p , q ), p ∈ s , q ∉ s from the two lists are iterated over, and an action a is taken if it satisfies a Metropolis-like criterion as detailed in ref . If no action is selected, the learning procedure terminates.…”