Reinforcement Learning in Different Phases of Quantum Control

Abstract

The ability to prepare a physical system in a desired quantum state is central to many areas of physics such as nuclear magnetic resonance, cold atoms, and quantum computing. Yet, preparing states quickly and with high fidelity remains a formidable challenge. In this work we implement cutting-edge Reinforcement Learning (RL) techniques and show that their performance is comparable to optimal control methods in the task of finding short, high-fidelity driving protocol from an initial to a target state in non-integrable many-body quantum systems of interacting qubits. RL methods learn about the underlying physical system solely through a single scalar reward (the fidelity of the resulting state) calculated from numerical simulations of the physical system. We further show that quantum state manipulation, viewed as an optimization problem, exhibits a spinglass-like phase transition in the space of protocols as a function of the protocol duration. Our RL-aided approach helps identify variational protocols with nearly optimal fidelity, even in the glassy phase, where optimal state manipulation is exponentially hard. This study highlights the potential usefulness of RL for applications in out-of-equilibrium quantum physics. S = {s = (t, h x (t))}, A = {a = δh x }, R = {r ∈ [0, 1]}.

Preparation of ordered states in ultra-cold gases using Bayesian optimization

“…These gradient based methods are considered as local optimization methods whose convergence is fast as long as the optimization landscape is well-behaved. However, these methods get compromised by the presence of any local minima and plateaus in the optimization landscape [52,53,54]. Alternatively, gradient-free algorithms are able to explore the optimization landscape more globally than gradient based methods making them less vulnerable to local minima.…”

“…It has the advantage of simplicity but can still get trapped in local minima and its convergence is limited by the presence of noise in the observations [51]. Other popular non-gradient methods include evolutionary algorithms [30,52,58] and the recently introduced reinforcement learning techniques [53,59,60]. These methods usually require large number of iterations to find the optimal solution.…”

Photonic architecture for reinforcement learning

“…Modern computing devices are rapidly evolving from handy resources to autonomous machines [1]. On the brink of this new technological revolution [2], reinforcement learning (RL) has emerged as a powerful and flexible tool to enable problem solving at an unprecedented scale, both in computer science [3-63-6] and in physics research [7][8][9][10][11][12][13]. This breakthrough development was in part spurred by the technological achievements of the last decades, which unlocked vast amounts of data and computational power.…”

Improving the dynamics of quantum sensors with reinforcement learning

“…In physics, many problems can be described within control theory which is concerned with finding a way to steer a system to achieve a goal [10]. The search for optimal control can naturally be formulated as reinforcement learning (RL) [11][12][13][14][15][16][17][18][19], a discipline of machine learning. RL has been used in the context of quantum control [17], to design experiments in quantum optics [20], and to automatically generate sequences of gates and measurements for quantum error correction [16,21,22].…”