Complex dynamic services and heterogeneous network environments make the asymmetrical control a curial issue to handle on the Internet. With the advent of the Internet of Things (IoT) and the fifth generation (5G), the emerging network applications lead to the explosive growth of mobile traffic while bringing forward more challenging service requirements to future radio access networks. Therefore, how to effectively allocate limited heterogeneous network resources to improve content delivery for massive application services to ensure network quality of service (QoS) becomes particularly urgent in heterogeneous network environments. To cope with the explosive mobile traffic caused by emerging Internet services, this paper designs an intelligent optimization strategy based on deep reinforcement learning (DRL) for resource allocation in heterogeneous cloud-edge-end collaboration environments. Meanwhile, the asymmetrical control problem caused by complex dynamic services and heterogeneous network environments is discussed and overcome by distributed cooperation among cloud-edge-end nodes in the system. Specifically, the multi-layer heterogeneous resource allocation problem is formulated as a maximal traffic offloading model, where content caching and request aggregation mechanisms are utilized. A novel DRL policy is proposed to improve content distribution by making cache replacement and task scheduling for arriving content requests in accordance with the information about users’ history requests, in-network cache capacity, available link bandwidth and topology structure. The performance of our proposed solution and its similar counterparts are analyzed in different network conditions.

With the rapid growth of Internet traffic and smart mobile terminals, ultradense networks are adopted as the key technology of the fifth generation to enhance resource utilization and content distribution while causing serious energy efficiency problem. Mobile edge computing has recently drawn great attention for its advantages in reducing transmission delay and network energy consumption by implementing caching and computing abilities at the edge of mobile networks. To improve network energy efficiency and content transmission, in this paper, we propose a novel energy-efficient hierarchical collaborative scheme by considering the in-network caching, request aggregation, and joint allocation of caching, computing, and communication resources in a layered heterogeneous network including mobile users, small base stations, macro base stations, and the cloud. We formulate the energy consumption problem as a queuing theory-based centralized model, where the same content requests can be aggregated in the queue of each base station. Then, the optimal solution is analyzed based on the distribution characteristic of content popularity at the base stations. Simulation results show that the performance of our proposed model is much better than the existing cloud-edge cooperation solutions without considering the deployment of caching resource and request aggregation policies.