A Low-Complexity Algorithm for a Reinforcement Learning-Based Channel Estimator for MIMO Systems

Abstract

This paper proposes a low-complexity algorithm for a reinforcement learning-based channel estimator for multiple-input multiple-output systems. The proposed channel estimator utilizes detected symbols to reduce the channel estimation error. However, the detected data symbols may include errors at the receiver owing to the characteristics of the wireless channels. Thus, the detected data symbols are selectively used as additional pilot symbols. To this end, a Markov decision process (MDP) problem is defined to optimize the selection of the detected data symbols. Subsequently, a reinforcement learning algorithm is developed to solve the MDP problem with computational efficiency. The developed algorithm derives the optimal policy in a closed form by introducing backup samples and data subblocks, to reduce latency and complexity. Simulations are conducted, and the results show that the proposed channel estimator significantly reduces the minimum-mean square error of the channel estimates, thus improving the block error rate compared to the conventional channel estimation.

Reinforcement Learning-Aided Channel Estimator in Time-Varying MIMO Systems

“…As a non-iterative approach, the reinforcement learning (RL)-aided channel estimator was introduced in [ 27 , 28 , 29 , 30 , 31 , 32 , 33 ]. The basic concept of this approach is the sequential selection of detected data symbols to minimize the channel estimation errors.…”

“…Hence, a Markov decision process (MDP) was defined to solve the sequential selection, and the corresponding optimal policy was derived in a closed-form expression in [ 31 ]. In [ 32 ], a low-complexity algorithm was investigated by introducing sub-blocks and finite backup samples, and the computational complexity and latency were significantly reduced without performance loss. Recently, a general framework for RL-aided channel estimation was studied in [ 33 ] based on Monte Carlo tree search.…”

“…Recently, a general framework for RL-aided channel estimation was studied in [ 33 ] based on Monte Carlo tree search. However, the RL-aided channel estimators in [ 31 , 32 , 33 ] were originally considered in time-invariant channels; they perform insufficiently in time-varying channels.…”

“…First, we define the optimization problem in time-varying channels to select the detected data symbols and minimize the estimation error between the estimated and current channels. This optimization problem is different from those in [ 31 , 32 , 33 ], where the selection of the detected data symbols is unchanged because the current channel remains unchanged with the time slot index. We propose an RL algorithm for the optimization problem that captures the time-varying nature of a channel.…”

“…Using this adjustment, we derive the optimal policy as a closed-form solution. Note that the proposed optimal policy differs from those in [ 31 , 32 , 33 ] because the influence of soft-decision symbols in the virtual state for future rewards gradually diminishes as the time slot index increases. We propose a further performance improvement scheme to refine the state elements.…”