A hierarchical model of data locality

Proceedings of the 23rd International Conference on Parallel Architectures and Compilation

Guttman

et al. 2014

Most of the prior compiler based data locality optimization works target exclusively cache locality optimization, and row-buffer locality in DRAM banks received much less attention. In particular, to the best of our knowledge, there is no single compiler based approach that can improve row-buffer locality in executing irregular applications. This presents a critical problem considering the fact that executing irregular applications in a power and performance efficient manner will be a key requirement to extract maximum benefits from emerging multicore machines and exascale systems. Motivated by these observations, this paper makes the following contributions. First, it presents a compiler-runtime cooperative data layout optimization approach that takes as input an irregular program that has already been optimized for cache locality and generates an output code with the same cache performance but better row-buffer locality (lower number of row-buffer misses). Second, it discusses a more aggressive strategy that sacrifices some cache performance in order to further improve row-buffer performance (i.e., it trades cache performance for memory system performance). The ultimate goal of this strategy is to find the right tradeoff point between cache performance and rowbuffer performance so that the overall application performance is improved. Third, the paper performs a detailed evaluation of these two approaches using both an AMD Opteron based multicore system and a multicore simulator. The experimental results, collected using five real-world irregular applications, show that (i) conventional cache optimizations do not improve row-buffer locality significantly; (ii) our first approach achieves about 9.8% execution time improvement by keeping the number of cache misses the same as a cache-optimized code but reducing the number of row-buffer misses; and (iii) our second approach achieves even higher execution time improvements (13.8% on average) by sacrificing cache performance for additional memory performance.

Section: Related Workmentioning

confidence: 99%

Trading cache hit rate for memory performance

Ding

Proceedings of the 23rd International Conference on Parallel Architectures and Compilation

Guttman

et al. 2014

“…Zhong et al [37] also employ a heuristic k-distance analysis in structure splitting and array regrouping according to this hierarchical model. Our locality group is different from the concept of "affinity group" proposed in [37,34]. Zhang et al [34] analyze the affinity model in detail and give the algorithmic complexity of finding reference affinity in the program trace.…”

Section: Related Workmentioning

confidence: 99%

“…Our locality group is different from the concept of "affinity group" proposed in [37,34]. Zhang et al [34] analyze the affinity model in detail and give the algorithmic complexity of finding reference affinity in the program trace. As opposed to [27], they target a common data access pattern rather than all patterns of data access.…”

Section: Related Workmentioning

confidence: 99%

“…However, most of these data locality oriented efforts are "heuristic approaches" since the locality optimization is a very hard problem in its general form [27]. In addition, much less effort has been spent in understanding the fundamental characteristics of data access patterns [34] exhibited by application programs and their interaction with underlying cache hardware [27].…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

CApRI

Ding

The 2014 ACM International Conference on Measurement and Modeling of Computer Systems

2014

Caches play a critical role in today's computer systems and optimizing their performance has been a critical objective in the last couple of decades. Unfortunately, compared to a plethora of work in software and hardware directed code/data optimizations, much less effort has been spent in understanding the fundamental characteristics of data access patterns exhibited by application programs and their interaction with the underlying cache hardware. Therefore, in general it is hard to reason about cache behavior of a program running on a target system. Motivated by this observation, we first set up a "locality model" that can help us determine the theoretical bounds of the cache misses caused by irregular data accesses. We then explain how this locality model can be used for different data locality optimization purposes. After that, based on our model, we propose a data reordering (data layout reorganization) scheme that can be applied after any existing data reordering schemes for irregular applications to improve cache performance by further reducing the cache misses. We evaluate the effectiveness of our scheme using a set of 8 programs with irregular data accesses, and show that it brings significant improvements over the state-of-the-art on two commercial multicore machines.

“…Zhang et al [52] evaluate the impact of cache sharing on parallel workloads. Zhang et al [51] study reference affinity and present a heuristic model for data locality. Chishti et al [15] propose a data mapping scheme that utilizes replication and capacity allocation techniques to improve locality.…”

Section: Related Workmentioning

confidence: 99%

Studying inter-core data reuse in multicores

Zhang

Proceedings of the ACM SIGMETRICS Joint International Conference on Measurement and Modeling of Computer Systems

Yemliha

2011

Most of existing research on emerging multicore machines focus on parallelism extraction and architectural level optimizations. While these optimizations are critical, complementary approaches such as data locality enhancement can also bring significant benefits. Most of the previous data locality optimization techniques have been proposed and evaluated in the context of single core architectures. While one can expect these optimizations to be useful for multicore machines as well, multicores present further opportunities due to shared on-chip caches most of them accommodate. In order to optimize data locality targeting multicore machines however, the first step is to understand data reuse characteristics of multithreaded applications and potential benefits shared caches can bring. Motivated by these observations, we make the following contributions in this paper. First, we give a definition for inter-core data reuse and quantify it on multicores using a set of ten multithreaded application programs. Second, we show that neither on-chip cache hierarchies of current multicore architectures nor state-ofthe-art (single-core centric) code/data optimizations exploit available inter-core data reuse in multithreaded applications. Third, we demonstrate that exploiting all available intercore reuse could boost overall application performance by around 21.3% on average, indicating that there is significant scope for optimization. However, we also show that trying to optimize for inter-core reuse aggressively without considering the impact of doing so on intra-core reuse can actually perform worse than optimizing for intra-core reuse alone. Finally, we present a novel, compiler-based data locality optimization strategy for multicores that balances both intercore and intra-core reuse optimizations carefully to maximize benefits that can be extracted from shared caches. Our experiments with this strategy reveal that it is very effective in optimizing data locality in multicores.