Reinforcement Learning with Neural Networks for Quantum Feedback

Abstract

Machine learning with artificial neural networks is revolutionizing science. The most advanced challenges require discovering answers autonomously. This is the domain of reinforcement learning, where control strategies are improved according to a reward function. The power of neural-networkbased reinforcement learning has been highlighted by spectacular recent successes, such as playing Go, but its benefits for physics are yet to be demonstrated. Here, we show how a network-based "agent" can discover complete quantum-error-correction strategies, protecting a collection of qubits against noise. These strategies require feedback adapted to measurement outcomes. Finding them from scratch, without human guidance, tailored to different hardware resources, is a formidable challenge due to the combinatorially large search space. To solve this, we develop two ideas: two-stage learning with teacher/student networks and a reward quantifying the capability to recover the quantum information stored in a multi-qubit system. Beyond its immediate impact on quantum computation, our work more generally demonstrates the promise of neural-network-based reinforcement learning in physics. arXiv:1802.05267v3 [quant-ph] 31 Aug 2018 stabilizer codes decoherence-free subspaces noise measurement qubits RL-environment neural network RL-agent action (gate)

Preparation of ordered states in ultra-cold gases using Bayesian optimization

“…It has the advantage of simplicity but can still get trapped in local minima and its convergence is limited by the presence of noise in the observations [51]. Other popular non-gradient methods include evolutionary algorithms [30,52,58] and the recently introduced reinforcement learning techniques [53,59,60]. These methods usually require large number of iterations to find the optimal solution.…”

Improving the dynamics of quantum sensors with reinforcement learning

“…In physics, many problems can be described within control theory which is concerned with finding a way to steer a system to achieve a goal [10]. The search for optimal control can naturally be formulated as reinforcement learning (RL) [11][12][13][14][15][16][17][18][19], a discipline of machine learning. RL has been used in the context of quantum control [17], to design experiments in quantum optics [20], and to automatically generate sequences of gates and measurements for quantum error correction [16,21,22].…”

Generalizable control for quantum parameter estimation through reinforcement learning

“…* Electronic address: hdyuan@mae.cuhk.edu.hk † Electronic address: x.wang@cityu.edu.hk Over the past few years, machine learning has demonstrated astonishing achievements in certain highdimensional input-output problems, such as playing video games [17] and mastering the game of Go [18]. Machine learning technics have been applied in physics covering many topics including experimental designs [19], finding optimal state transfer schemes in a spin chain [20] and discovering the quantum-error-correction strategies under noises [21]. Among the machine learning algorithms, reinforcement learning is one of the most actively researched [22].…”