Low-Cost Epoch-Based Correlation Prefetching for Commercial Applications

Abstract

The performance of many important commercial workloads, such as on-line transaction processing, is limited by the frequent stalls due to off-chip instruction and data accesses. These applications are characterized by irregular control flow and complex data access patterns that render many low-cost prefetching schemes, such as stream-based and stride-based prefetching, ineffective. For such applications, correlation-based prefetching, which is capable of capturing complex data access patterns, has been shown to be a more promising approach. However, the large instruction and data working sets of these applications require extremely large correlation tables, making these tables impractical to be implemented on-chip. This paper proposes the epoch-based correlation prefetcher, which cost-effectively stores its correlation table in main memory and exploits the concept of epochs to hide the long latency of its correlation table access, and which attempts to eliminate entire epochs instead of individual instruction and data misses. Experimental results demonstrate that the epoch-based correlation prefetcher, which requires minimal on-chip real estate to implement, improves the performance of a suite of important commercial benchmarks by 13% to 31% and significantly outperforms previously proposed correlation prefetchers.40th IEEE/ACM International Symposium on Microarchitecture

Temporal streams in commercial server applications

“…Furthermore, prior studies have considered only a single system organization, focusing either on uniprocessors [7,8,9,21] or multi-chip distributed-shared-memory systems [25]. As we will show, server applications' off-chip miss behavior changes drastically when all cores are located on a single chip, because coherence activity and contention are captured entirely within that chip.…”

“…Temporal streams extend the notion of address correlation to sequences rather than pairs of misses. Although the term "temporal stream" was introduced in [25], a wide variety of recent prefetchers rely on the same underlying phenomenon, including hot data stream prefetching [7], the global history buffer [19], the user-level memory thread [21], epoch-based correlation prefetching [8], and last-touch correlated data streaming [9].…”

“…Off-chip memory accesses continue to pose a critical performance bottleneck in commercial server applications [1,3,8,22,25]. Extensive research has shown that widely-deployed stride-based prefetchers provide only limited benefit for many server applications, such as online transaction processing and web serving, because these applications are dominated by pointer-based data structures with complex, non-strided access patterns [8,22,25].…”

“…To improve performance for these access patterns, over a decade of research has lead to the development of address-correlating prefetchers, which exploit correlation between consecutive memory accesses and are highly-effective for pointerbased structures [6,7,8,9,13,14,15,19,21,25]. Building on this line of research, the latest proposals prefetch extended sequences of memory accesses that recur over the course of program execution [7,9,19,21,25].…”

MLP-Aware Dynamic Instruction Window Resizing in Superscalar Processors for Adaptively Exploiting Available Parallelism

“…Prefetchers [1], [2] are generally inexpensive but are effective only for regular access patterns. Some methods that are effective for irregular patterns have been documented in the literature, but these are very costly [3] because the correlation between irregular accesses must be recorded.…”