Reinforcement Learning with Quantum Variational Circuit

Abstract

The development of quantum computational techniques has advanced greatly in recent years, parallel to the advancements in techniques for deep reinforcement learning. This work explores the potential for quantum computing to facilitate reinforcement learning problems. Quantum computing approaches offer important potential improvements in time and space complexity over traditional algorithms because of its ability to exploit the quantum phenomena of superposition and entanglement. Specifically, we investigate the use of quantum variational circuits, a form of quantum machine learning. We present our techniques for encoding classical data for a quantum variational circuit, we further explore pure and hybrid quantum algorithms for DQN and Double DQN. Our results indicate both hybrid and pure quantum variational circuit have the ability to solve reinforcement learning tasks with a smaller parameter space. These comparison are conducted with two OpenAI Gym environments: CartPole and Blackjack, The success of this work is indicative of a strong future relationship between quantum machine learning and deep reinforcement learning.

A Survey on Quantum Reinforcement Learning

“…To demonstrate the basic functionality of the model, initial experiments are conducted on the CartPole environment. The results demonstrate a similar performance to Lockwood and Si [LS20]. On the two Atari environments, the paper considers 12 different hybrid architectures (dense vs. convolutional encoding, 5 vs. 10 vs. 15 qubits, dense vs. pooling decoding), which are compared to a well-established classical architecture.…”

“…Continuous states are scaled to the finite interval [−π/2, +π/2] by applying arctan to the raw observations. The result serves as the rotation angle for an R x rotation, which is very similar to the scaled encoding proposed by Lockwood and Si [LS20]. In order to increase expressivity w.r.t.…”

“…Playing Atari with Hybrid Quantum-Classical Reinforcement Learning, Lockwood and Si (2021) Summary. This work by Lockwood and Si [LS21] extends their previous paper [LS20], which, in turn, was based on Chen et al [Che+20], where Q-learning with VQC function approximation has been introduced. The paper considers the Atari environments Pong and Breakout, with continuous state space of dimensionality 28.224 (the observations are cropped and converted to images with 84×84×4 pixels).…”

“…Summary. The work by Lockwood and Si [LS20] modifies several aspects of the routine proposed by Chen et al [Che+20]. Most importantly, they introduce two new encoding schemes to deal with a continuous state space.…”

Generation of Truly Random Numbers on a Quantum Annealer

“…In recent years, a gate-based system (IBM Q) and an adiabatic quantum annealer (D-Wave System) have become available for research and commercial purposes. Both gate-based models [38], [40] and annealers [8], [23], [52], [53] have recently been employed for pattern recognition.…”