Precise Runahead Execution

Abstract

Runahead execution improves processor performance by accurately prefetching long-latency memory accesses. When a long-latency load causes the instruction window to fill up and halt the pipeline, the processor enters runahead mode and keeps speculatively executing code to trigger accurate prefetches. A recent improvement tracks the chain of instructions that leads to the long-latency load, stores it in a runahead buffer, and executes only this chain during runahead execution, with the purpose of generating more prefetch requests during runahead execution. Unfortunately, all these prior runahead proposals have shortcomings that limit performance and energy efficiency because they discard the full instruction window to enter runahead mode and then flush the pipeline to restart normal operation. This significantly constrains the performance benefits and increases the energy overhead of runahead execution. In addition, runahead buffer limits prefetch coverage by tracking only a single chain of instructions that lead to the same long-latency load. We propose precise runahead execution (PRE) to mitigate the shortcomings of prior work. PRE leverages the renaming unit to track all the dependency chains leading to long-latency loads. PRE uses a novel approach to manage free processor resources to execute the detected instruction chains in runahead mode without flushing the pipeline. Our results show that PRE achieves an additional 21.1% performance improvement over the recent runahead proposals while reducing energy consumption by 6.1%.

Vector Runahead for Indirect Memory Accesses

“…Hardware prefetchers can pick up a variety of memoryaccess patterns, but to achieve the instruction-level visibility necessary to calculate the addresses of complex access patterns in today's workloads [1], one must operate within the core, instead of within the cache. Runahead execution [8,9] is the most promising technique to achieve this.…”

“…First, by skipping over loads for which the data source is not yet ready, it is unsuitable for today's complex indirection patterns that consist of chains of dependent load misses. Second, conventional runahead is limited by both the processor's front-end (fetch/decode/rename) width and available back-end resources (issue queue slots and physical registers) [9]. What is needed is a technique that can overcome the limitations of a processor's resources to generate massive amounts of memory-level parallelism and follow chains of dependent loads to completion, prefetching all data required for many memory accesses in the future.…”

“…What is needed is a technique that can overcome the limitations of a processor's resources to generate massive amounts of memory-level parallelism and follow chains of dependent loads to completion, prefetching all data required for many memory accesses in the future. Vector (b) Precise Runahead Execution (PRE) [9] is able to prefetch array elements from A. In contrast, the array elements to B cannot be prefetched during runahead mode as they depend on A.…”

“…Fig. 2: Vector Runahead versus Precise Runahead Execution (PRE) [9] on an illustrative code example. The loads highlighted in green can only be triggered by stalling on loads highlighted in gray, and those in blue by stalling on gray and green.…”

“…On a variety of graph, database and high-performance computing workloads, Vector Runahead improves performance by 1.79× compared to a baseline out-of-order processor with a stride prefetcher. Relative to the state-of-the-art Indirect Memory Prefetcher (IMP) [12] and Precise Runahead Execution (PRE) [9], Vector Runahead improves performance by 1.49× on average. The fundamental reason for this significant performance improvement is illustrated in Figure 1: PRE is unable to accurately prefetch the majority of indirect memory accesses, unlike Vector Runahead.…”