Taking Gradients Through Experiments: LSTMs and Memory Proximal Policy Optimization for Black-Box Quantum Control

August, Moritz; Hernández-Lobato, José Miguel

doi:10.1007/978-3-030-02465-9_43

Cited by 38 publications

(40 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In this paper, we adopt a radically different approach to this problem based on machine learning (ML) [40][41][42][43][44][45][46]. ML has recently been applied successfully to several problems in equilibrium condensed matter physics [47,48], turbulent dynamics [49,50] and experimental design [51,52], and here we demonstrate that Reinforcement Learning (RL) provides deep insights into nonequilibrium quantum dynamics [53][54][55][56][57][58]. Specifically, we use a modified version of the Watkins Q-Learning algorithm [40] to teach a computer agent to find driving protocols which prepare a quantum system in a target state |ψ * starting from an initial state |ψ i by controlling a time-dependent field.…”

Section: Introductionmentioning

confidence: 99%

Reinforcement Learning in Different Phases of Quantum Control

Bukov¹,

Day²,

Sels³

et al. 2018

Phys. Rev. X

392

335

View full text Add to dashboard Cite

The ability to prepare a physical system in a desired quantum state is central to many areas of physics such as nuclear magnetic resonance, cold atoms, and quantum computing. Yet, preparing states quickly and with high fidelity remains a formidable challenge. In this work we implement cutting-edge Reinforcement Learning (RL) techniques and show that their performance is comparable to optimal control methods in the task of finding short, high-fidelity driving protocol from an initial to a target state in non-integrable many-body quantum systems of interacting qubits. RL methods learn about the underlying physical system solely through a single scalar reward (the fidelity of the resulting state) calculated from numerical simulations of the physical system. We further show that quantum state manipulation, viewed as an optimization problem, exhibits a spinglass-like phase transition in the space of protocols as a function of the protocol duration. Our RL-aided approach helps identify variational protocols with nearly optimal fidelity, even in the glassy phase, where optimal state manipulation is exponentially hard. This study highlights the potential usefulness of RL for applications in out-of-equilibrium quantum physics. S = {s = (t, h x (t))}, A = {a = δh x }, R = {r ∈ [0, 1]}.

show abstract

Section: Introductionmentioning

confidence: 99%

Reinforcement Learning in Different Phases of Quantum Control

Bukov¹,

Day²,

Sels³

et al. 2018

Phys. Rev. X

392

335

View full text Add to dashboard Cite

show abstract

“…In the mean time, numerically, the state preparation paradigm has been formulated as an optimisation problem [44][45][46][47][48][49][50][51][52]. Recently, stochastic descent, gradientbased GRAPE [53] and CRAB [54], and model-free Machine Learning [48,[55][56][57][58][59][60][61][62][63][64][65][66] have proven useful algorithms to find approximate fast-forward Hamiltonians in singleparticle and many-body systems.…”

Section: Introductionmentioning

confidence: 99%

Geometric Speed Limit of Accessible Many-Body State Preparation

2019

View full text Add to dashboard Cite

We analyze state preparation within a restricted space of local control parameters between adiabatically connected states of control Hamiltonians. We formulate a conjecture that the time integral of energy fluctuations over the protocol duration is bounded from below by the geodesic length set by the quantum geometric tensor. The conjecture implies a geometric lower bound for the quantum speed limit (QSL). We prove the conjecture for arbitrary, sufficiently slow protocols using adiabatic perturbation theory and show that the bound is saturated by geodesic protocols, which keep the energy variance constant along the trajectory. Our conjecture implies that any optimal unit-fidelity protocol, even those that drive the system far from equilibrium, are fundamentally constrained by the quantum geometry of adiabatic evolution. When the control space includes all possible couplings, spanning the full Hilbert space, we recover the well-known Mandelstam-Tamm bound. However, using only accessible local controls to anneal in complex models such as glasses or to target individual excited states in quantum chaotic systems, the geometric bound for the quantum speed limit can be exponentially large in the system size due to a diverging geodesic length. We validate our conjecture both analytically by constructing counter-diabatic and fast-forward protocols for a three-level system, and numerically in nonintegrable spin chains and a nonlocal SYK model. arXiv:1804.05399v2 [quant-ph]

show abstract

“…Unfortunately, these theories have limited application in non-integrable manybody systems, for which no exact closed-form expressions can be obtained. This has motivated the development of efficient numerical algorithms, such as GRAPE [16,17], CRAB [18], and Machine learning based approaches [19][20][21][22][23][24][25][26][27][28][29][30][31]. State preparation can be formulated as an optimal control problem for which the objective is to find the set of controls that extremize a cost function, i.e.…”

mentioning

confidence: 99%

Glassy Phase of Optimal Quantum Control

et al. 2019

View full text Add to dashboard Cite

We study the problem of preparing a quantum many-body system from an initial to a target state by optimizing the fidelity over the family of bang-bang protocols. We present compelling numerical evidence for a universal spin-glass-like transition controlled by the protocol time duration. The glassy critical point is marked by a proliferation of protocols with close-to-optimal fidelity and with a true optimum that appears exponentially difficult to locate. Using a machine learning (ML) inspired framework based on the manifold learning algorithm t-SNE, we are able to visualize the geometry of the high-dimensional control landscape in an effective low-dimensional representation. Across the transition, the control landscape features an exponential number of clusters separated by extensive barriers, which bears a strong resemblance with replica symmetry breaking in spin glasses and random satisfiability problems. We further show that the quantum control landscape maps onto a disorder-free classical Ising model with frustrated nonlocal, multibody interactions. Our work highlights an intricate but unexpected connection between optimal quantum control and spin glass physics, and shows how tools from ML can be used to visualize and understand glassy optimization landscapes.Establishing the general limitations and constraints of quantum control is crucial for guiding the field forward.Recently, it was shown that the quantum state preparation paradigm supports a number of control phase transitions by varying the protocol duration T [22, 32, 33], exhibiting overconstrained, controllable, correlated, and glassy phases. Glass-like systems are expected to feature slow equilibration time scales related to an underlying

show abstract

Taking Gradients Through Experiments: LSTMs and Memory Proximal Policy Optimization for Black-Box Quantum Control

Cited by 38 publications

References 26 publications

Reinforcement Learning in Different Phases of Quantum Control

Reinforcement Learning in Different Phases of Quantum Control

Geometric Speed Limit of Accessible Many-Body State Preparation

Glassy Phase of Optimal Quantum Control

Contact Info

Product

Resources

About