This paper presents High density heterogeneous networks (HetNet) which are the most promising technology for the fifth generation (5G) cellular network. Since 5G will be available for a long time, previous generation networking systems will need customization and updates. We examine the merits and drawbacks of legacy and Q-Learning (QL)-based adaptive resource allocation systems. Furthermore, various comparisons between methods and schemes are made for the purpose of evaluating the solutions for future generation. Microwave macro cells are used to enable extra high capacity such as Long-Term Evolution (LTE), eNodeB (eNB), and Multimedia Communications Wireless technology (MC), in which they are most likely to be deployed. This paper also presents four scenarios for 5G mm-Wave implementation, including proposed system architectures. The WL algorithm allocates optimal power to the small cell base station (SBS) to satisfy the minimum necessary capacity of macro cell user equipment (MUEs) and small cell user equipment (SCUEs) in order to provide quality of service (QoS) (SUEs). The challenges with dense HetNet and the massive backhaul traffic they generate are discussed in this study. Finally, a core HetNet design based on clusters is aimed at reducing backhaul traffic. According to our findings, MM-wave HetNet and MEC can be useful in a wide range of applications, including ultra-high data rate and low latency communications in 5G and beyond. We also used the channel model simulator to examine the directional power delay profile with received signal power, path loss, and path loss exponent (PLE) for both LOS and NLOS using uniform linear array (ULA) 2X2 and 64x16 antenna configurations at 38 GHz and 73 GHz mmWave bands for both LOS and NLOS (NYUSIM). The simulation results show the performance of several path loss models in the mmWave and sub-6 GHz bands. The path loss in the close-in (CI) model at mmWave bands is higher than that of open space and two ray path loss models because it considers all shadowing and reflection effects between transmitter and receiver. We also compared the suggested method to existing models like Amiri, Su, Alsobhi, Iqbal, and greedy (non adaptive), and found that it not only enhanced MUE and SUE minimum capacities and reduced BT complexity, but it also established a new minimum QoS threshold. We also talked about 6G researches in the future. When compared to utilizing the dual slope route loss model alone in a hybrid heterogeneous network, our simulation findings show that decoupling is more visible when employing the dual slope path loss model, which enhances system performance in terms of coverage and data rate.