Driving black-box quantum thermal machines with optimal power/efficiency trade-offs using reinforcement learning

Abstract

The optimal control of non-equilibrium open quantum systems is a challenging task but has a key role in improving existing quantum information processing technologies. We introduce a general model-free framework based on Reinforcement Learning to identify out-of-equilibrium thermodynamic cycles that are Pareto optimal trade-offs between power and efficiency for quantum heat engines and refrigerators. The method does not require any knowledge of the quantum thermal machine, nor of the system model, nor of the quantum state. Instead, it only observes the heat fluxes, so it is both applicable to simulations and experimental devices. We test our method identifying Pareto-optimal trade-offs between power and efficiency in two systems: an experimentally realistic refrigerator based on a superconducting qubit, where we identify non-intuitive control sequences that reduce quantum friction and outperform previous cycles proposed in literature; and a heat engine based on a quantum harmonic oscillator, where we find cycles with an elaborate structure that outperform the optimized Otto cycle.

Quantum optimal control in quantum technologies. Strategic report on current status, visions and goals for research in Europe

“…QOCT for heat engines has been almost exclusively applied to the discrete Carnot and Otto four stroke cycles. Typical optimization targets are maximum efficiency, maximum power, or minimal fluctuations [217,582,641]. A trade-off has been identified between these tasks.…”

Self-correcting quantum many-body control using reinforcement learning with tensor networks

“…While MPS-based algorithms have been used in the context of optimal many-body control to find high-fidelity protocols [17][18][19][20] , the advantages of deep RL for quantum control 21 have so far been investigated using exact simulations of only a small number of interacting quantum degrees of freedom. Nevertheless, policy-gradient and value-function RL algorithms have recently been established as useful tools in the study of quantum state preparation [22][23][24][25][26][27][28][29][30][31][32][33][34][35][36][37][38][39] , quantum error correction and mitigation [40][41][42][43] , quantum circuit design [44][45][46][47] , quantum metrology 48,49 , and quantum heat engines 50,51 ; quantum reinforcement learning algorithms have been proposed as well [52][53][54][55][56] . Thus, in times of rapidly developing quantum simulators which exceed the computational capabilities of classical computers 57 , the natural question arises regarding scaling up the size of quantum systems in RL control studies beyond exact diagonalization methods.…”

Quantum optimal control in quantum technologies. Strategic report on current status, visions and goals for research in Europe

“…QOCT for heat engines has been almost exclusively applied to the discrete Carnot and Otto four stroke cycles. Typical optimization targets are maximum efficiency, maximum power, or minimal fluctuations [214,577,635]. A trade-off has been identified between these tasks.…”