Deep Reinforcement Learning on FPGA for Self-Healing Cryogenic Power Amplifier Control

Abstract

Wireless sensing and communication for space exploration in areas inaccessible to human often suffer from severe performance degradation due to the cryogenic effects on the transmitters' circuits. To survive extreme temperatures, programmable radio frequency (RF) power amplifiers (PA) can be built into the transmitter, and intelligent PA controllers need to be integrated into the system to interact with the environment and restore the PA's functionalities. This problem can be modeled as the controller acts (control the PA) in an environment to maximize the reward (signal quality), and it is most suitable to use reinforcement learning as a solution. This paper presents a cryogenic and energy-efficient reinforcement learning (RL) module on Field Programmable Gate Arrays (FPGA) that can directly program the PA. By characterizing a self-healing PA in a liquid nitrogen environment, we generated an RF signal data set and built an interactive RL environment to model the PA's behaviors across its configurations and cryogenic temperatures down to −197 • C. We developed a deep RL model with a high generalization capability introduced by the neural networks to control the PA and restore its performance. The RL model with fixed-point training and inference is implemented on FPGA to survive the cryogenic conditions and carry out fast and low-power training and inference for PA control. All functionalities of the programmed FPGA operate correctly in the cryogenic testing environment.INDEX TERMS Deep reinforcement learning, quantized neural networks, quantized training, fixed-point arithmetic, FPGA, power amplifier, cryogenic circuits.