Resource constraints in Long Term Evolution-Advanced (LTE-A)/5G heterogeneous networks pose significant challenges to maintaining high-quality and real-time data transmission. Quality of Service (QoS) is crucial for ensuring user satisfaction across both real-time (RT) and non-real-time (NRT) applications. This paper proposes a novel scheduling and resource allocation scheme that employs a Smoothed Round-Robin (SRR) algorithm to classify traffic into real-time (RT) and non-real-time (NRT) classes. A power-constrained resource allocation method based on Deep Q-learning (DQL) is then applied to manage these traffic classes. Furthermore, we propose a handover mechanism that utilizes the Weighted Aggregated Sum Product Assessment (WASPAS) method to address mobility and inter-cell interference challenges. Simulation results demonstrate the superior performance of the proposed scheme compared to existing solutions, showcasing improvements in delay, throughput, fairness index, call drop rate, and packet loss rate. This research presents a novel, efficient approach to QoS-aware resource allocation in LTE-A/5G HetNets.