The unprecedented growth in data demands for 5G communication systems necessitates advanced techniques to maximize spectral efficiency while ensuring user fairness and low latency. This study proposes Adaptive Dual-Layer Resource Allocation (ADLRA), a novel hybrid technique combining Non-Orthogonal Multiple Access (NOMA) and Rate-Splitting Multiple Access (RSMA). The ADLRA framework introduces dynamic user pairing, hierarchical beamforming, and adaptive power and rate allocation strategies to optimize resource utilization.
Key features include dynamic user pairing, leveraging machine learning algorithms for efficient group formation based on channel conditions, and hierarchical beamforming, which prioritizes high-priority users in the NOMA layer while effectively managing shared resources in the RSMA layer. Interference mitigation is achieved through spatial filtering and multi-user diversity techniques, ensuring minimal intra-cell and inter-cell interference. Simulation results demonstrate significant performance gains Spectral efficiency improved by 32%, compared to traditional NOMA. Latency reduced by 18%, ensuring seamless communication for ultra-reliable low-latency applications. Achieved a 94% fairness index, reflecting equitable resource allocation among users. Enhanced throughput, with an average gain of 28%, compared to RSMA-only systems. These results highlight the potential of ADLRA to meet the stringent requirements of next-generation 5G systems, offering a scalable and efficient solution for diverse communication scenarios. The proposed method sets a foundation for future hybrid access strategies in wireless communication networks.