Caching in Base Station with Recommendation via Q-Learning

Guo, Kaiyang; Yang, Chenyang; Liu, Tingting

doi:10.1109/wcnc.2017.7925848

Cited by 48 publications

(42 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…• Energy-Minimum Algorithm (EMA): In the system model, each user is associated with the closest SBS, and the SBS chooses the most local popular contents in its cache [8].…”

Section: E Performance Comparisonmentioning

confidence: 99%

See 1 more Smart Citation

Energy and Delay Optimization for Cache-Enabled Dense Small Cell Networks

Chen

2020

IEEE Trans. Veh. Technol.

View full text Add to dashboard Cite

Caching popular files in small base stations (SBSs) has been proved to be an effective way to reduce bandwidth pressure on the backhaul links of dense small cell networks (DSCNs). Many existing studies on cache-enabled DSCNs attempt to improve user experience by optimizing end-to-end file delivery delay. However, under practical scenarios where files (e.g., video files) have diverse quality of service requirements, energy consumption at SBSs should also be concerned from the network perspective. In this paper,we attempt to optimize these two critical metrics in cache-enabled DSCNs. Firstly, we formulate the energy-delay optimization problem as a Mixed Integer Programming (MIP) problem, where file placement, user association and power control are jointly considered. To model the tradeoff relationship between energy consumption and end-to-end file delivery delay, a utility function linearly combining these two metrics is used as an objective function of the optimization problem. Then, we solve the problem in two stages, i.e. caching stage and delivery stage, based on the observation that caching is performed during off-peak time. At the caching stage, a local popular file placement policy is proposed by estimating user preference at each SBS. At the delivery stage, with given caching status at SBSs, the MIP problem is further decomposed by Benders' decomposition method. An efficient algorithm is proposed to approach the optimal association and power solution by iteratively shrinking the gap of the upper and lower bounds. Finally, extension simulations are performed to validate our analytical and algorithmic work. The results demonstrate that the proposed algorithms can achieve the optimal tradeoff between energy consumption and end-to-end file delivery delay.

show abstract

“…• Energy-Minimum Algorithm (EMA): In the system model, each user is associated with the closest SBS, and the SBS chooses the most local popular contents in its cache [8].…”

Section: E Performance Comparisonmentioning

confidence: 99%

“…In cache-enabled DSCNs, user experience is also improved due to the reduction of end-to-end file delivery delay [1] [5]- [8]. When a user requests a file cached in the local SBS, the file is delivered by that SBS instead of the faraway Internet file server.…”

Section: Introductionmentioning

confidence: 99%

Energy and Delay Optimization for Cache-Enabled Dense Small Cell Networks

Chen

2020

IEEE Trans. Veh. Technol.

View full text Add to dashboard Cite

show abstract

“…Due to the fact that context information of users is private and the historical request number of a single user is often very limited, online estimation of user preference is very difficult in practice. Therefore, some works try to make caching decisions directly, rather than estimating user preference first and then optimizing cache, to minimize the long-term cost or maximize the long-term reward by using reinforcement learning [25], [26] and deep reinforcement leaning [27]. However, these works [25]- [27] only focus on the singlecell performance and no coordination among multiple SBSs exists.…”

Section: Introductionmentioning

confidence: 99%

Collaborative Multi-Agent Multi-Armed Bandit Learning for Small-Cell Caching

Tao

Shen

2020

IEEE Trans. Wireless Commun.

View full text Add to dashboard Cite

This paper investigates learning-based caching in small-cell networks (SCNs) when user preference is unknown. The goal is to optimize the cache placement in each small base station (SBS) for minimizing the system long-term transmission delay. We model this sequential multi-agent decision making problem in a multi-agent multi-armed bandit (MAMAB) perspective. Rather than estimating user preference first and then optimizing the cache strategy, we propose several MAMAB-based algorithms to directly learn the cache strategy online in both stationary and non-stationary environment. In the stationary environment, we first propose two high-complexity agent-based collaborative MAMAB algorithms with performance guarantee. Then we propose a low-complexity distributed MAMAB which ignores the SBS coordination. To achieve a better balance between SBS coordination gain and computational complexity, we develop an edge-based collaborative MAMAB with the coordination graph edge-based reward assignment method. In the non-stationary environment, we modify the MAMAB-based algorithms proposed in the stationary environment by proposing a practical initialization method and designing new perturbed terms to adapt to the dynamic environment. Simulation results are provided to validate the effectiveness of our proposed algorithms. The effects of different parameters on caching performance are also discussed.

show abstract

“…However, the users' context and content feature information exploited in [10], [11] are often unavailable if the cache node is operated by mobile network operators, which in general can only observe local content requests. The work [12] integrates recommendation with local caching in wireless edge and proposes a reinforcement learning (RL) based algorithm to optimize the cache replacement policy. A RL-based algorithm is also proposed in [13] to find the optimal policy in an online fashion, enabling the cache node to track the space-time popularity dynamics.…”

Section: Introductionmentioning

confidence: 99%

Using Grouped Linear Prediction and Accelerated Reinforcement Learning for Online Content Caching

Zhang

Zheng

Tao

2018

2018 IEEE International Conference on Communications Workshops (ICC Workshops)

View full text Add to dashboard Cite

Proactive caching is an effective way to alleviate peak-hour traffic congestion by prefetching popular contents at the wireless network edge. To maximize the caching efficiency requires the knowledge of content popularity profile, which however is often unavailable in advance. In this paper, we first propose a new linear prediction model, named grouped linear model (GLM) to estimate the future content requests based on historical data. Unlike many existing works that assumed the static content popularity profile, our model can adapt to the temporal variation of the content popularity in practical systems due to the arrival of new contents and dynamics of user preference. Based on the predicted content requests, we then propose a reinforcement learning approach with model-free acceleration (RLMA) for online cache replacement by taking into account both the cache hits and replacement cost. This approach accelerates the learning process in non-stationary environment by generating imaginary samples for Q-value updates. Numerical results based on real-world traces show that the proposed prediction and learning based online caching policy outperform all considered existing schemes.

show abstract

Caching in Base Station with Recommendation via Q-Learning

Cited by 48 publications

References 15 publications

Energy and Delay Optimization for Cache-Enabled Dense Small Cell Networks

Energy and Delay Optimization for Cache-Enabled Dense Small Cell Networks

Collaborative Multi-Agent Multi-Armed Bandit Learning for Small-Cell Caching

Using Grouped Linear Prediction and Accelerated Reinforcement Learning for Online Content Caching

Contact Info

Product

Resources

About