Reinforcement Learning vs. Gradient-Based Optimisation for Robust Energy Landscape Control of Spin-1/2 Quantum Networks

Abstract

We explore the use of policy gradient methods in reinforcement learning for quantum control via energy landscape shaping of XX-Heisenberg spin chains in a model agnostic fashion. Their performance is compared to finding controllers using gradient-based L-BFGS optimisation with restarts, with full access to an analytical model. Hamiltonian noise and coarse-graining of fidelity measurements are considered. Reinforcement learning is able to tackle challenging, noisy quantum control problems where L-BFGS optimization algorithms struggle to perform well. Robustness analysis under different levels of Hamiltonian noise indicates that controllers found by reinforcement learning appear to be less affected by noise than those found with L-BFGS.

Quantum optimal control in quantum technologies. Strategic report on current status, visions and goals for research in Europe

“…A recent systematic comparison has shown for qubit manipulation that RL outperforms standard optimization procedures when the problem is discretized and the space of the action is sufficiently small [658]. RL can be also efficiently combined with gradient-based optimization procedures for the robust control of spin 1/2 networks against different noise sources [328].…”

Fast Quantum Gate Design with Deep Reinforcement Learning Using Real-Time Feedback on Readout Signals

“…Reinforcement learning [21][22][23] is an alternative approach for gate design which operates without prior knowledge of the hardware model. Reinforcement learning and its variants have been applied to myriad quantum control problems using numerical simulated environments [24][25][26][27][28][29][30][31][32][33][34][35][36][37][38][39][40][41]. Such set-ups demonstrate the potential of reinforcement learning, but suffer from the same modelling constraints as other methods.…”

Information Transfer in Spintronics Networks under Worst Case Uncertain Parameter Errors

“…( 4) is complicated with many local maxima/minima, making the accurate solution for the optimal parameters ∆ and t a challenging task. In [10], the authors modify convex optimization methods to maximize the fidelity subject to model uncertainties and in [11] Khalid et al use modelagnostic reinforcement learning methods to explore the complex fidelity surface and obtain high fidelity solutions.…”