Self-Correcting Quantum Many-Body Control using Reinforcement Learning with Tensor Networks

Abstract

Quantum many-body control is a central milestone en route to harnessing quantum technologies. However, the exponential growth of the Hilbert space dimension with the number of qubits makes it challenging to classically simulate quantum many-body systems and consequently, to devise reliable and robust optimal control protocols. Here, we present a novel framework for efficiently controlling quantum many-body systems based on reinforcement learning (RL). We tackle the quantum control problem by leveraging matrix product states (i) for representing the many-body state and, (ii) as part of the trainable machine learning architecture for our RL agent. The framework is applied to prepare ground states of the quantum Ising chain, including critical states. It allows us to control systems far larger than neural-network-only architectures permit, while retaining the advantages of deep learning algorithms, such as generalizability and trainable robustness to noise. In particular, we demonstrate that RL agents are capable of finding universal controls, of learning how to optimally steer previously unseen many-body states, and of adapting control protocols on-the-fly when the quantum dynamics is subject to stochastic perturbations.

Driving black-box quantum thermal machines with optimal power/efficiency trade-offs using reinforcement learning

“…RL has received great attention for its success at mastering tasks beyond human-level such as playing games [82][83][84], and for robotic applications [85]. RL has been recently used for quantum control [86][87][88][89][90][91][92][93], outperforming previous state-of-the-art methods [94,95], for fault-tolerant quantum computation [96,97], and to minimize entropy production in closed quantum systems [98].…”

Monte Carlo Tree Search based Hybrid Optimization of Variational Quantum Circuits

“…Quantum control and variational quantum eigensolver: Traditional optimal quantum control methods, often used in prior works, are GRAPE (Khaneja et al, 2005) and CRAB (Caneva et al, 2011). More recently, success has been seen by the combination of traditional methods with machine learning (Schäfer et al, 2020;Wang et al, 2020a;Sauvage and Mintert, 2019;Fösel et al, 2020;Nautrup et al, 2019;Albarrán-Arriagada et al, 2018;Sim et al, 2021;Wu et al, 2020a,b;Anand et al, 2020;Dalgaard et al, 2022), and especially reinforcement learning (Niu et al, 2019;Fösel et al, 2018;August and Hernández-Lobato, 2018;Porotti et al, 2019;Wauters et al, 2020;Yao et al, 2020a;Sung, 2020;Chen et al, 2013;Bukov, 2018;Sørdal and Bergli, 2019;Bolens and Heyl, 2020;Dalgaard et al, 2020;Metz and Bukov, 2022)). Among them, Variational quantum eigensolver or VQE (Cerezo et al, 2021a;Tilly et al, 2021) provides a general framework applicable on noisy intermediate-scale quantum (NISQ) devices (Preskill, 2018) to variationally tune the circuit parameters and improve the approximation.…”

Optimal steering of matrix product states and quantum many-body scars

“…Typically, in such numerical setups one optimizes the fidelity of the state preparation over the trajectory in the multidimensional space of control parameters using gradientfree [19] or gradient-based routines [20][21][22][23]. In addition, machine-learning based approaches to this problem were also considered [24,25].…”