Cache-conscious structure layout

Abstract

Hardware trends have produced an increasing disparity between processor speeds and memory access times. While a variety of techniques for tolerating or reducing memory latency have been proposed, these are rarely successful for pointer-manipulating programs.This paper explores a complementary approach that attacks the source (poor reference locality) of the problem rather than its manifestation (memory latency). It demonstrates that careful data organization and layout provides an essential mechanism to improve the cache locality of pointer-manipulating programs and consequently, their performance. It explores two placement technique-lustering and colorinet improve cache performance by increasing a pointer structure's spatial and temporal locality, and by reducing cache-conflicts.To reduce the cost of applying these techniques, this paper discusses two strategies-cache-conscious reorganization and cacheconscious allocation--and describes two semi-automatic toolsccmorph and ccmalloc-that use these strategies to produce cache-conscious pointer structure layouts. ccmorph is a transparent tree reorganizer that utilizes topology information to cluster and color the structure. ccmalloc is a cache-conscious heap allocator that attempts to co-locate contemporaneously accessed data elements in the same physical cache block. Our evaluations, with microbenchmarks, several small benchmarks, and a couple of large real-world applications, demonstrate that the cache-conscious structure layouts produced by ccmorph and ccmalloc offer large performance benefit-n most cases, significantly outperforming state-of-the-art prefetching.

Effective jump-pointer prefetching for linked data structures

“…implemented cache specific techniques such as compression and line coloring for LDS nodes in a memory allocator [3] and a generational garbage collector [4]. One of the earliest software-controlled LDS prefetching scheme was SPAID of Lipasti et.…”

Optimizing graph algorithms for improved cache performance

“…Chilimbi et al discusses making pointerbased data structures cache-conscious in [8]. They focus on providing structure layouts to make tree structures cache-conscious.…”

“…A number of groups are attempting to improve performance by performing computations in memory [6] [25]. Other groups are attacking the problem in software; either in the compiler through reordering instructions and prefetching [18] [19][20] [36] or through complex data layouts to improve cache performance [8][10] [15].…”

A prefetching technique for irregular accesses to linked data structures

“…Cache conscious data placement has been explored in [4,5,9]. The technique tries to pack LDS nodes that are accessed in a row into the same or consecutive cache blocks.…”