On Resource Pooling and Separation for LRU Caching

Abstract

Caching systems using the Least Recently Used (LRU) principle have now become ubiquitous. A fundamental question for these systems is whether the cache space should be pooled together or divided to serve multiple flows of data item requests in order to minimize the miss probabilities. In this paper, we show that there is no straight yes or no answer to this question, depending on complex combinations of critical factors, including, e.g., request rates, overlapped data items across different request flows, data item popularities and their sizes. Specifically, we characterize the asymptotic miss probabilities for multiple competing request flows under resource pooling and separation for LRU caching when the cache size is large.Analytically, we show that it is asymptotically optimal to jointly serve multiple flows if their data item sizes and popularity distributions are similar and their arrival rates do not differ significantly; the self-organizing property of LRU caching automatically optimizes the resource allocation among them asymptotically. Otherwise, separating these flows could be better, e.g., when data sizes vary significantly. We also quantify critical points beyond which resource pooling is better than separation for each of the flows when the overlapped data items exceed certain levels. Technically, we generalize existing results on the asymptotic miss probability of LRU caching for a broad class of heavy-tailed distributions and extend them to multiple competing flows with varying data item sizes, which also validates the Che approximation under certain conditions. These results provide new insights on improving the performance of caching systems.

Asymptotic Miss Ratio of LRU Caching with Consistent Hashing

“…However, the overall conditional asymptotic miss ratio of the cluster is always the same almost surely, depending on the probability distribution of the random hash function. Due to this equivalence, we comment that the engineering implications discussed in [9] for a single LRU cache can also be extended to a LRU cluster.…”

“…and using Lemma 1 in [9], we obtain, for ∀ 1 > 0, there exists a large x 2 such that for all x > x 2 ,…”

“…To this end, we use Lemmas 5, 6 and 7 to study the three quantities in Lemma 1, i.e., q i /q i+1 , Φ(·) and T (x). Specifically, we first note that the variables I Lemma 1 is a direct consequence of Theorem 1 in [9]. Let the data item popularity distribution on a server be (q i , i ≥ 1).…”