Probabilistic shared cache management (PriSM)

Abstract

Effective sharing of the last level cache has a significant influence on the overall performance of a multicore system. We observe that existing solutions control cache occupancy at a coarser granularity, do not scale well to large core counts and in some cases lack the flexibility to support a variety of performance goals. In this paper, we propose Probabilistic Shared Cache Management (PriSM), a framework to manage the cache occupancy of different cores at cache block granularity by controlling their eviction probabilities. The proposed framework requires only simple hardware changes to implement, can scale to larger core count and is flexible enough to support a variety of performance goals. We demonstrate the flexibility of PriSM, by computing the eviction probabilities needed to achieve goals like hit-maximization, fairness and QOS. PriSM-HitMax improves performance by 18.7% over LRU and 11.8% over previously proposed schemes in a sixteen core machine. PriSM-Fairness improves fairness over existing solutions by 23.3% along with a performance improvement of 19.0%. PriSM-QOS successfully achieves the desired QOS targets.

Understanding object-level memory access patterns across the spectrum

“…Implications Many prior studies on hardware architecture, memory hierarchy design or program access pattern analysis observe and test small sample segments from program execution, with typical lengths of under 200 million instructions [17,31,33,41] and 200-500 million instructions [20,25,36,46,50,58] Our experiments reveal that one or a few such samples often fail to capture a full picture of the programs' memory access behavior, even within their "steady phase" (after initialization and before nalization).…”

Last level cache partitioning via multiverse thread classification

“…Manikantan et al proposed a framework that computes eviction probabilities for each core and replaces the cache lines according to these probabilities in order to achieve a finer granularity at line level [10]. Probabilistic shared cache management (PriSM) collects the augmented cache hit information using shadow tags and obtains target size.…”

A Practical Data Classification Framework for Scalable and High Performance Chip-Multiprocessors

“…Heap data blocks are also identified as practically private if at least one pointer in the reference list is of global scope (i.e., the pointer is global or passed to other threads) and that pointer is dereferenced with a TI variable. 1 Typically, this indicates that these data blocks are probably private. In many embarrassingly parallel applications, probably private is actually private when the global data blocks are exclusively partitioned among multiple threads by TI variables.…”

“…These factors include remote data access latency, coherence overhead, inter-core communication penalty, etc. While many effective improvements [1]- [6] to CMPs have been proposed to address these issues, many optimization opportunities are lost unless specific characteristics of the application(s) running on the system can be leveraged. In particular, the lack of information about characteristics of a running application such as data access patterns, communication behavior, memory usage, etc., typically results in a more generic but less efficient method for utilizing on-chip caches, interconnect and coherence directories.…”