Realizing a deep reinforcement learning agent discovering real-time feedback control strategies for a quantum system

Reuer, Kevin; Landgraf, Jonas; Fösel, T.; O'Sullivan, James; Beltrán, Liberto; Akın, Abdulkadir; Norris, Graham J.; Remm, Ants; Kerschbaum, M.; Besse, Jean-Claude; Marquardt, Florian; Wallraff, Andreas; Eichler, Christopher

doi:10.48550/arxiv.2210.16715

Cited by 2 publications

(1 citation statement)

References 53 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…While MPS-based algorithms have been used in the context of optimal many-body control to find high-fidelity protocols [17][18][19][20] , the advantages of deep RL for quantum control 21 have so far been investigated using exact simulations of only a small number of interacting quantum degrees of freedom. Nevertheless, policy-gradient and value-function RL algorithms have recently been established as useful tools in the study of quantum state preparation [22][23][24][25][26][27][28][29][30][31][32][33][34][35][36][37][38][39] , quantum error correction and mitigation [40][41][42][43] , quantum circuit design [44][45][46][47] , quantum metrology 48,49 , and quantum heat engines 50,51 ; quantum reinforcement learning algorithms have been proposed as well [52][53][54][55][56] . Thus, in times of rapidly developing quantum simulators which exceed the computational capabilities of classical computers 57 , the natural question arises regarding scaling up the size of quantum systems in RL control studies beyond exact diagonalization methods.…”

Section: State-informed Many-body Controlmentioning

confidence: 99%

Self-correcting quantum many-body control using reinforcement learning with tensor networks

Metz

Bukov

2023

Nat Mach Intell

View full text Add to dashboard Cite

Quantum many-body control is a central milestone en route to harnessing quantum technologies. However, the exponential growth of the Hilbert space dimension with the number of qubits makes it challenging to classically simulate quantum many-body systems and, consequently, to devise reliable and robust optimal control protocols. Here we present a framework for efficiently controlling quantum many-body systems based on reinforcement learning (RL). We tackle the quantum-control problem by leveraging matrix product states (1) for representing the many-body state and (2) as part of the trainable machine learning architecture for our RL agent. The framework is applied to prepare ground states of the quantum Ising chain, including states in the critical region. It allows us to control systems far larger than neural-network-only architectures permit, while retaining the advantages of deep learning algorithms, such as generalizability and trainable robustness to noise. In particular, we demonstrate that RL agents are capable of finding universal controls, of learning how to optimally steer previously unseen many-body states and of adapting control protocols on the fly when the quantum dynamics is subject to stochastic perturbations. Furthermore, we map our RL framework to a hybrid quantum–classical algorithm that can be performed on noisy intermediate-scale quantum devices and test it under the presence of experimentally relevant sources of noise.

show abstract