In this paper, we propose a long short-term memory-based deep learning (DL) architecture for signal detection in uplink and downlink rate-splitting multiple access systems with multi-carrier modulation, over Nakagami-m fading and generalized Gaussian noise (GGN). The proposed DL detector completely eliminates the need for the use of successive interference cancellation (SIC), which suffers from disadvantages such as error propagation. In an orthogonal frequency division multiplexing setting, we show that the proposed DL detector outperforms the standard SIC receivers such as the least squares detector and the minimum mean-squared error receiver, and attains the performance of the optimal maximum likelihood detector, in terms of the symbol error rate (SER). Furthermore, we study the effects of the shaping parameter of GGN, hyperparameters of the DL network such as batch size and learning rate on the SER performance. INDEX TERMS Deep learning (DL), generalized Gaussian noise (GGN), long short-term memory (LSTM), non-orthogonal multiple access (NOMA), rate-splitting multiple access (RSMA).ANAGHA K. KOWSHIK (Graduate Student Member, IEEE) received the bachelor's degree (B.

No abstract

No abstract

No abstract

No abstract