The dynamic resource block structure (D-RBS) allows for flexible allocation of radio resources. This flexibility (potentially) enables efficient utilization of available resources and adaptability to changing network conditions. In this context, managing resource contention and optimizing allocation decisions become increasingly challenging. In this research paper, we introduce a new approach for D-RBS for re-allocation of enhanced Mobile Broadband (eMBB) and massive Machine Type Communication (mMTC) resource blocs (RBs) to URLLC users. Our scheme leverages artificial intelligence (AI) to support the three main services of 5th generation networks. To efficiently allocate resources for eMBB/mMTC and URLLC services, we propose an intelligent puncturing scheme. Additionally, we formulate an optimization problem that aims to minimize resource and transmit power usage while meeting the requirements of all three services. Since this problem is non-convex and involves multiple optimization variables, we utilize deep reinforcement learning as a solution algorithm. We then compare our proposed intelligent allocation (IA) scheme with two other schemes: random allocation (RaA) and overallocation (OA), which have lower complexity and overhead. Performance and complexity analyses are conducted in a multi-cell scenario with interference. Our results demonstrate that the IA scheme outperforms RaA and OA, achieving an energy efficiency gain of 40% and 15% respectively. However, it is worth noting that IA has a 36.3% higher complexity in terms of action selection compared to RaA and OA.INDEX TERMS Dynamic RB structure, URLLC puncturing, eMBB and URLLC multiplexing, MA-DRL.