A Reinforcement Learning-Based User-Assisted Caching Strategy for Dynamic Content Library in Small Cell Networks

Abstract

This paper studies the problem of joint edge cache placement and content delivery in cache-enabled small cell networks in the presence of spatio-temporal content dynamics unknown a priori. The small base stations (SBSs) satisfy users' content requests either directly from their local caches, or by retrieving from other SBSs' caches or from the content server. In contrast to previous approaches that assume a static content library at the server, this paper considers a more realistic nonstationary content library, where new contents may emerge over time at different locations. To keep track of spatio-temporal content dynamics, we propose that the new contents cached at users can be exploited by the SBSs to timely update their flexible cache memories in addition to their routine off-peak main cache updates from the content server. To take into account the variations in traffic demands as well as the limited caching space at the SBSs, a user-assisted caching strategy is proposed based on reinforcement learning principles to progressively optimize the caching policy with the target of maximizing the weighted network utility in the long run. Simulation results verify the superior performance of the proposed caching strategy against various benchmark designs.

Optimal Load-Splitting and Distributed-Caching for Dynamic Content Over the Wireless Edge

“…Based on Proposition 4 and according to the value iteration algorithm, the optimal policy that solves (22) has a threshold structure given in Equation (23). Next, we show that the thresholds η m , ∀m are uniformly bounded.…”

A Survey on Reinforcement Learning-Aided Caching in Heterogeneous Mobile Edge Networks

“…The optimization of content caching and delivery policy under non-stationary content libraries through a user-assisted RL algorithm is the subject of [71]. The network utility consists of backhaul traffic offloading, cache hit rate, content retrieval and delivery.…”

Contextual Learning for Content Caching With Unknown Time-Varying Popularity Profiles via Incremental Clustering

“…Content caching can be applied in many different types of communication systems from cellular networks (e.g., [6]- [8]) to device-to-device networks (e.g., [9], [10]). Many researchers focus on investigating content caching styles including deterministic, probabilistic and coded caching.…”

“…After updating the Q-value, the agent moves to the next state and executes the action which has been chosen earlier. The Q-value for a state-action is updated by 8) where ω represents the learning rate which determines to what extent the Q-value is updated based on the newly acquired information and γ is the discount factor.…”