Quantum Tree-Based Planning

Abstract

Reinforcement Learning is at the core of a recent revolution in the Artificial Intelligence community. Simultaneously, we are witnessing the emergence of a new field: Quantum Machine Learning. In this work, we reach for the interplay between Quantum Computing and Reinforcement Learning. Learning by interaction is possible in the quantum setting using the concept of oraculization of environments. The paper extends previous oracular instances to address more general stochastic environments. In this setting, we developed a novel quantum algorithm for near-optimal decision-making based on the Reinforcement Learning paradigm known as Sparse Sampling. The proposed algorithm exhibits a quadratic speedup compared with its classical counterpart. To the best of the authors' knowledge, this is the first quantum planning algorithm exhibiting a time complexity independent of the number of states of the environment, which makes it suitable for large state space environments, where planning is otherwise intractable.

Policy gradients using variational quantum circuits

“…Previous results suggest that RL agents obeying the rules of quantum mechanics can outperform classical RL agents (Dunjko et al 2016(Dunjko et al , 2017Paparo et al 2014;Sequeira et al 2021;Dunjko and Briegel 2018;Saggio et al 2021). However, these suffer from the same scaling problem as classical tabular RL: they do not scale easily to real-world problems with large state-action spaces.…”

“…Variational quantum circuits (VQCs) are a viable alternative since state-action pairs can be parameterized, enabling, at least in theory, a reduction in the circuit's complexity. Moreover, VQCs could enable shallow enough circuits to be confidently executed on current NISQ (Noisy Intermediate Scale Quantum) hardware (Preskill 2018) without resorting to typical brute force search over the state/action space as in the quantum tabular setting (Sequeira et al 2021;Dunjko et al 2016). Variational models are also referred to as approximately universal quantum neural networks (Farhi and Neven 2018;Schuld et al 2021).…”