Deep learning integrated reinforcement learning for adaptive beamforming in B5G networks

Abstract

In this paper, a deep learning integrated reinforcement learning (DLIRL) algorithm is proposed for comprehending intelligent beamsteering in Beyond Fifth Generation (B5G) networks. The smart base station in B5G networks aims to steer the beam towards appropriate user equipment based on the acquaintance of isotropic transmissions. The foremost methodology is to optimize beam direction through reinforcement learning that delivers significant improvement in signal to noise ratio (SNR). This includes alternate path finding during path obstruction and steering the beam appropriately between the smart base station and user equipment. The DLIRL is realized through supervised learning with deep neural networks and deep Q-learning schemes. The proposed algorithm comprises of an online learning phase for training the weights and a working phase for carrying out the prediction. Results confirm that the performance of the B5G system is improved considerably as compared to its counterparts with a spectral efficiency of 11 bps/Hz at SNR = 10 dB for a bit error rate performance of 10 −5 . As compared to reinforced learning and deep neural network with a deviation of ±3 o and ±5 • , respectively, the DLIRL beamforming displays a deviation of ±2 o . Moreover, the DLIRL can track the user equipment and steer the beam in its direction with an accuracy of 92%.

A Literature Survey on AI-Aided Beamforming and Beam Management for 5G and 6G Systems

“…On the other hand, Elbir et al [169] generate the precoder from artificial channels using a convolutional neural network, achieving better results than the heuristic, deep learning, and Multilayer Perceptron (MLP) solutions that were compared in the article. However, samples of real-world network indicators are abundant in most cases, such as AoA and Angle of Departure (AoD) [170], the pilots present in different frame configurations [171,172], and samples from the channel [173] and, therefore, can also be used to train neural networks and result in more accurate precoders, tailored to specific conditions.…”

“…The authors propose a new way of combining DL and RL for beamforming, leveraging high spectral efficiency and overall beamforming effectiveness. [172] ine Dynamic subarrays AHC Proposed hybrid precoding, which can efficiently avoid mutually correlated metrics.…”

A Literature Survey on AI-Aided Beamforming and Beam Management for 5G and 6G Systems

“…On the other hand, Elbir et al [164] generated precoder from artificial channels using a convolutional neural network, achieving better results than the heuristic, deep learning, and Multilayer Perceptrons (MLP) solutions that were compared in the article. However, samples of real-world network indicators are abundant in most cases, such as AoA and Angle of Departure (AoD) [165], the pilots present in different frame configurations [166,167], and samples from the channel [168], and, therefore, can also be used to train neural networks and result in more accurate precoders, tailored to specific conditions.…”

4. Towards Joint Communication and Sensing

“…RL is then to optimize the beam direction, whose state space, action space, reward, and learning algorithm are defined in [5]. Initial simulation is performed on MATLAB 2021a siteviewer for radio transmission (RT) using site viewer and simultaneous channel generation [6], but it could equally well have used Winprop, which models more accurate geometries and accommodates more surface properties [5]. The output of RT is fed for channel generation, which in turn produces input for the DNN for training.…”

“…If a beamformer steers the main beam in a particular direction (θ , ϕ), then the directional accuracy is defined as ±δθ, ±δϕ from the maximum signal-to-noise ratio (SNR) of the main lobe. The proposed deep learning integrated reinforcement learning (DLIRL) beamforming has an angle-of-departure (AoD) accuracy towards the UE location with a deviation of ±2 • , whereas RL has a deviation of ±3 • and DNN's deviation is ±5 • [6] is required to measure the ToA at accuracy levels of ∼33 picoseconds (calculated from 1 cm/speed of light).…”