Enlarging Instruction Streams

Abstract

Abstract-The stream fetch engine is a high-performance fetch architecture based on the concept of an instruction stream. We call a sequence of instructions from the target of a taken branch to the next taken branch, potentially containing multiple basic blocks, a stream. The long length of instruction streams makes it possible for the stream fetch engine to provide a high fetch bandwidth and to hide the branch predictor access latency, leading to performance results close to a trace cache at a lower implementation cost and complexity. Therefore, enlarging instruction streams is an excellent way to improve the stream fetch engine. In this paper, we present several hardware and software mechanisms focused on enlarging those streams that finalize at particular branch types. However, our results point out that focusing on particular branch types is not a good strategy due to Amdahl's law. Consequently, we propose the multiple-stream predictor, a novel mechanism that deals with all branch types by combining single streams into long virtual streams. This proposal tolerates the prediction table access latency without requiring the complexity caused by additional hardware mechanisms like prediction overriding. Moreover, it provides high-performance results which are comparable to state-of-the-art fetch architectures but with a simpler design that consumes less energy.

Temporal instruction fetch streaming

“…Similar repetition in the sequence of basic blocks visited by a program has been reported by Larus [16] and underlies trace scheduling [9] and trace caches [26]. Temporal instruction streams differ from previously-defined instruction streams [22,23,27] in two key respects: (1) temporal instruction streams are defined at cache-block rather than basic-block granularity, and (2) temporal instruction streams span fetch discontinuities. TIFS efficiently records and exploits long recurring temporal instruction streams.…”

“…Although computer architectures and workloads have evolved drastically since then, simple next-line instruction prefetching remains critical to the performance of modern commercial server workloads [17]. More recent work on instruction-stream prefetching generalizes the notion of the next-line instruction prefetcher to arbitrary-length sequences of contiguous basic blocks [23,27].…”

“…We show that nearly all instruction-cache misses, including control transfers to noncontiguous instructions, occur as part of such recurring sequences, which we call temporal instruction streams. In contrast to the instruction streams in [22,23,27], which comprise a sequence of contiguous basic blocks, temporal instruction streams comprise recurring sequences of instruction-cache blocks that span discontinuities (taken branches) within the instruction address sequence. We record temporal instruction streams as they are encountered, and replay them to predict misses, eliminating the inefficiencies that arise through reconstructing the sequences indirectly with branch predictions.…”

Shift

“…Simple next-line instruction prefetchers varying in prefetch degree have been ubiquitously employed in commercial processors to eliminate misses to subsequent blocks [28,32,33] and fail to eliminate instruction cache misses due to discontinuities in the program control flow caused by function calls, taken branches and interrupts.…”