Experimental Deep Reinforcement Learning for Error-Robust Gate-Set Design on a Superconducting Quantum Computer

Abstract

Quantum computers promise tremendous impact across applications -and have shown great strides in hardware engineering -but remain notoriously error prone. Careful design of low-level controls has been shown to compensate for the processes which induce hardware errors, leveraging techniques from optimal and robust control. However, these techniques rely heavily on the availability of highly accurate and detailed physical models which generally only achieve sufficient representative fidelity for the most simple operations and generic noise modes. In this work, we use deep reinforcement learning to design a universal set of error-robust quantum logic gates on a superconducting quantum computer, without requiring knowledge of a specific Hamiltonian model of the system, its controls, or its underlying error processes. We experimentally demonstrate that a fully autonomous deep reinforcement learning agent can design single qubit gates up to 3× faster than default DRAG operations without additional leakage error, and exhibiting robustness against calibration drifts over weeks. We then show that ZX(−π/2) operations implemented using the cross-resonance interaction can outperform hardware default gates by over 2× and equivalently exhibit superior calibration-free performance up to 25 days post optimization using various metrics. We benchmark the performance of deep reinforcement learning derived gates against other black box optimization techniques, showing that deep reinforcement learning can achieve comparable or marginally superior performance, even with limited hardware access.

Summary: Chicago Quantum Exchange (CQE) Pulse-level Quantum Control Workshop

“…The work [60] proposes a black-box approach to designing an error-robust universal quantum gate set using a deep reinforcement learning (DRL) model, as shown in the inset of Fig. 7(a).…”

Numerical Gate Synthesis for Quantum Heuristics on Bosonic Quantum Processors

“…However, these methods are currently tested on specific limited cases, and insights are difficult to generalize, e.g. see [15][16][17]. In this paper we follow the techniques described in Ref.…”

Gradient Ascent Pulse Engineering with Feedback

“…This included both pure control tasks (e.g. [28][29][30][31], even in an experiment [32]) but in particular also the more challenging quantum real-time feedback tasks that rely on adaptive responses to measurement outcomes [33][34][35][36].…”

“…On the one hand, this can be an advantage in applying it to experimental setups whose parameters are partially unknown (as emphasized e.g. in [32,35]). On the other hand, much of the training time is spent in learning (implicitly) a model of the dynamics while simultaneously attempting to find good feedback strategies.…”