Optimal Dynamic Proactive Caching Via Reinforcement Learning

Sadeghi, Alireza; Sheikholeslami, Fatemeh; Giannakis, Georgios B.

doi:10.1109/spawc.2018.8445899

Cited by 12 publications

(10 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To model the time-varying nature of task popularity, the popularity profile t is modeled by a V -state Markov chain [22] , represented by V different popularity profiles .1/ ; : : : ; .V / . Each profile is modeled by Zipf distributions with parameters Á v .…”

Section: Task Modelmentioning

confidence: 99%

“…The objective function in P1 is modified by using Eqs. (17) and (22) under the RL framework. The constraints in P1 are used during the training of DRL framework to determine whether to adopt or discard a certain learning result.…”

Section: Drl-based Solution With Ddpgmentioning

confidence: 99%

“…Similar to the definition of reward given in Eq. (22), the overall cost J s t for s-th cell in Eq. (31) can be expressed by the reward function R s , which maps the state-action pair to a scalar reward r s t , such that…”

Section: Rewardmentioning

confidence: 99%

“…The fetching cost of each task is assumed to be the same as the data size of corresponding task, i.e., g k D b k . Moreover, the popularity profile t of the tasks is modeled by a three-state Markov chain [22] , represented by three different popularity profiles .1/ ; .2/ , and .3/ . These profiles are modeled by Zipf distributions with parameters Á 1 D 1; Á 2 D 1:2; and Á 3 D 1:5, respectively.…”

Section: Rewardmentioning

confidence: 99%

See 3 more Smart Citations

Deep reinforcement learning for dynamic computation offloading and resource allocation in cache-assisted mobile edge computing systems

Nath

2020

Intell. and Converged Netw.

164

View full text Add to dashboard Cite

Section: Task Modelmentioning

confidence: 99%

Section: Drl-based Solution With Ddpgmentioning

confidence: 99%

Section: Rewardmentioning

confidence: 99%

Section: Rewardmentioning

confidence: 99%

See 2 more Smart Citations

Deep reinforcement learning for dynamic computation offloading and resource allocation in cache-assisted mobile edge computing systems

Nath

2020

Intell. and Converged Netw.

164

View full text Add to dashboard Cite

“…Denoting the long-term popularity of the content as p := E[r t ], using the expressions for the optimal actions in (13a)-(13d), and leveraging the independence among r t , λ t , and ρ t , the expected cost-to-go function can be readily derived as in (14)- (16). The expectation in (14) is w.r.t.…”

Section: Value Function In Closed Formmentioning

confidence: 99%

Reinforcement Learning for Adaptive Caching With Dynamic Storage Pricing

Sadeghi

Sheikholeslami

Marqués

et al. 2019

IEEE J. Select. Areas Commun.

Self Cite

View full text Add to dashboard Cite

Small base stations (SBs) of fifth-generation (5G) cellular networks are envisioned to have storage devices to locally serve requests for reusable and popular contents by caching them at the edge of the network, close to the end users. The ultimate goal is to shift part of the predictable load on the backhaul links, from on-peak to off-peak periods, contributing to a better overall network performance and service experience. To enable the SBs with efficient fetch-cache decision-making schemes operating in dynamic settings, this paper introduces simple but flexible generic time-varying fetching and caching costs, which are then used to formulate a constrained minimization of the aggregate cost across files and time. Since caching decisions per time slot influence the content availability in future slots, the novel formulation for optimal fetch-cache decisions falls into the class of dynamic programming. Under this generic formulation, first by considering stationary distributions for the costs and file popularities, an efficient reinforcement learningbased solver known as value iteration algorithm can be used to solve the emerging optimization problem. Later, it is shown that practical limitations on cache capacity can be handled using a particular instance of the generic dynamic pricing formulation. Under this setting, to provide a light-weight online solver for the corresponding optimization, the well-known reinforcement learning algorithm, Q-learning, is employed to find optimal fetchcache decisions. Numerical tests corroborating the merits of the proposed approach wrap up the paper.

show abstract

Deep learning-based edge caching for multi-cluster heterogeneous networks

Yang

Zhang

et al. 2019

Neural Comput & Applic

View full text Add to dashboard Cite

In this work, we consider a time and space evolution cache refreshing in multicluster heterogeneous networks. We consider a two-step content placement probability optimization. At the initial complete cache refreshing optimization, the joint optimization of the activated base station density and the content placement probability is considered. And we transform this optimization problem into a GP problem. At the following partial cache refreshing optimization, we take the time-space-evolution into consideration and derive a convex optimization problem subjected to the cache capacity constraint and the backhaul limit constraint. We exploit the redundant information in different content popularity using the Deep Neural Network to avoid the repeated calculation because of the change of content popularity distribution at different time slot.

show abstract

Optimal Dynamic Proactive Caching Via Reinforcement Learning

Cited by 12 publications

References 12 publications

Deep reinforcement learning for dynamic computation offloading and resource allocation in cache-assisted mobile edge computing systems

Deep reinforcement learning for dynamic computation offloading and resource allocation in cache-assisted mobile edge computing systems

Reinforcement Learning for Adaptive Caching With Dynamic Storage Pricing

Deep learning-based edge caching for multi-cluster heterogeneous networks

Contact Info

Product

Resources

About