Attention Based Neural Networks for Wireless Channel Estimation

Luan, Dianxin; Thompson, John F.

doi:10.48550/arxiv.2204.13465

Cited by 4 publications

(10 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Attention mechanism has been employed to help the DLbased communication system, such as channel state information (CSI) compression [20], channel compression [21] and CE [22] [16]. It can enhance the estimation accuracy for channels with highly separate distributions in [16].…”

Section: B Attention Mechanismmentioning

confidence: 99%

Learn to Adapt to New Environments From Past Experience and Few Pilot Blocks

Wang

Gao

2023

IEEE Trans. Cogn. Commun. Netw.

View full text Add to dashboard Cite

In recent years, deep learning has been widely applied in communications and achieved remarkable performance improvement. Most of the existing works are based on data-driven deep learning, which requires a significant amount of training data for the communication model to adapt to new environments and results in huge computing resources for collecting data and retraining the model. In this paper, we will significantly reduce the required amount of training data for new environments by leveraging the learning experience from the known environments. Therefore, we introduce few-shot learning to enable the communication model to generalize to new environments, which is realized by an attention-based method. With the attention network embedded into the deep learningbased communication model, environments with different power delay profiles can be learnt together in the training process, which is called the learning experience. By exploiting the learning experience, the communication model only requires few pilot blocks to perform well in the new environment. Through an example of deep-learning-based channel estimation, we demonstrate that this novel design method achieves better performance than the existing data-driven approach designed for few-shot learning.

show abstract

Section: B Attention Mechanismmentioning

confidence: 99%

Learn to Adapt to New Environments From Past Experience and Few Pilot Blocks

Wang

Gao

2023

IEEE Trans. Cogn. Commun. Netw.

View full text Add to dashboard Cite

show abstract

“…This will ensure that the subsequent neural network can focus on the critical features to yield the best channel estimate. Compared to HA02 [22], the neural architecture of the encoder is modified for the Channelformer to provide both reduced complexity and improved performance.…”

Section: B Main Contributions and Outlinementioning

confidence: 99%

“…Compared with ReEsNet, Interpolation-ResNet with only 9,442 parameters (called InterpolateNet in this paper) [10] achieves a slightly improved performance and 82% reduced parameters. However, the generalization capability of InterpolateNet and ReEsNet trained [10] is quite limited, which motivates us to research with the attention mechanism to propose HA02 [22]. As the other neural network solutions are much less complex than ChannelNet, we only consider the stateof-the-art networks when presenting simulation results.…”

Section: Channelnet Reesnet Tr and Interpolation-resnetmentioning

confidence: 99%

Channelformer: Attention based Neural Solution for Wireless Channel Estimation and Effective Online Training

Luan¹,

Thompson²

2023

Preprint

View full text Add to dashboard Cite

In this paper, we propose an encoder-decoder neural architecture (called Channelformer) to achieve improved channel estimation for orthogonal frequency-division multiplexing (OFDM) waveforms in downlink scenarios. The self-attention mechanism is employed to achieve input precoding for the input features before processing them in the decoder. In particular, we implement multi-head attention in the encoder and a residual convolutional neural architecture as the decoder, respectively. We also employ a customized weight-level pruning to slim the trained neural network with a fine-tuning process, which reduces the computational complexity significantly to realize a low complexity and low latency solution.This enables reductions of up to 70% in the parameters, while maintaining an almost identical performance compared with the complete Channelformer. We also propose an effective online training method based on the fifth generation (5G) new radio (NR) configuration for the modern communication systems, which only needs the available information at the receiver for online training. Using industrial standard channel models, the simulations of attention-based solutions show superior estimation performance compared with other candidate neural network methods for channel estimation.

show abstract

“…A transformer network can be designed to process variable lengths of sequences as inputs and outputs, which is beneficial for designing scalable channel estimators. However, several transformer-based estimators, including in Li and Peng [20] and Luan and Thompson [22], might not process variable size of resource grids through fully connected layers and/or upscaling modules, whose input and output dimensions should be predetermined to a specific configuration. Moreover, as shown in the later part of this paper, the baseline transformer-based estimator tends to learn a specific scenario in the training dataset rather than general relationships, like other baseline DL-based estimators.…”

Section: Related Workmentioning

confidence: 99%

“…Self-attention mechanism and transformer network [16] have great successes in various NLP applications, for example, Devlin and others [17], by its powerful feature extractions, and also for various image processing applications [18]. Several works [19][20][21][22] presented channel estimations based on transformer network or attention mechanisms. A transformer network can be designed to process variable lengths of sequences as inputs and outputs, which is beneficial for designing scalable channel estimators.…”

Section: Related Workmentioning

confidence: 99%

Deep learning‐based scalable and robust channel estimator for wireless cellular networks

et al. 2022

View full text Add to dashboard Cite

In this paper, we present a two‐stage scalable channel estimator (TSCE), a deep learning (DL)‐based scalable, and robust channel estimator for wireless cellular networks, which is made up of two DL networks to efficiently support different resource allocation sizes and reference signal configurations. Both networks use the transformer, one of cutting‐edge neural network architecture, as a backbone for accurate estimation. For computation‐efficient global feature extractions, we propose using window and window averaging‐based self‐attentions. Our results show that TSCE learns wireless propagation channels correctly and outperforms both traditional estimators and baseline DL‐based estimators. Additionally, scalability and robustness evaluations are performed, revealing that TSCE is more robust in various environments than the baseline DL‐based estimators.

show abstract

Attention Based Neural Networks for Wireless Channel Estimation

Cited by 4 publications

References 14 publications

Learn to Adapt to New Environments From Past Experience and Few Pilot Blocks

Learn to Adapt to New Environments From Past Experience and Few Pilot Blocks

Channelformer: Attention based Neural Solution for Wireless Channel Estimation and Effective Online Training

Deep learning‐based scalable and robust channel estimator for wireless cellular networks

Contact Info

Product

Resources

About