Scaling the issue window with look-ahead latency prediction

Abstract

In contemporary out-of-order superscalar design, high IPC is mainly achieved by exposing high instruction level parallelism (ILP). Scaling issue window size can certainly provide more ILP; however, future processor scaling demands threaten to limit the size of the issue window.In this study, we propose a dynamic instruction sorting mechanism that provides more ILP without increasing the size of the issue window. In our approach, early in the pipeline, we predict how long an instruction needs to wait before it can be issued, i.e. the waiting time for its operands to be produced. Using this knowledge, the instructions are placed into a sorting structure, which allows instructions with shorter waiting times enter the issue window ahead of those instructions with longer waiting times, preventing long-waiting instructions from clogging the issue queue.The accuracy in predicting instruction waiting times directly determines the effectiveness of our sorting mechanism. While most instructions have deterministic execution latencies, predicting load execution times is more difficult due to cache misses and in-flight loads. Loads are particularly challenging since their execution time can vary significantly. In this study, we examine techniques to predict load execution time accurately, based on data reference history.

Exploiting Operand Availability for Efficient Simultaneous Multithreading

“…Other proposals have introduced new scheduling techniques with the goal of designing scalable dynamic schedulers to support a very large number of in-flight instructions [3], [12], [23], [24], [28]. Brown et al [4] proposed removing the selection logic from the critical path by exploiting the fact that the number of ready instructions in a given cycle is typically smaller than the processor's issue width.…”

“…Scheduling techniques based on predicting the issue cycle of an instruction [1], [8], [9], [16], [19], [24], [26], [36] remove the wake-up delay from the critical path and remove the CAM logic from instruction wake-up, but need to keep track of the cycle when each physical register will become ready. In [13], the wake-up time prediction occurs in parallel with the instruction fetching.…”

Reducing the energy of speculative instruction schedulers

“…As shown in our prior work [14], this nondeterminism comes from cache misses, loads that alias with in-flight data blocks, and memory bus contention. Load instructions can take anywhere from several cycles to several hundred cycles in current generation processors.…”

Efficient Instruction Schedulers for SMT Processors

“…Other proposals have introduced new scheduling techniques with the goal of designing scalable dynamic schedulers to support a very large number of in-flight instructions [3,12,23,24,28]. Brown et al [4] proposed to remove the selection logic from the critical path by exploiting the fact that the number of ready instructions in a given cycle is typically smaller than the processor's issue width.…”

“…Scheduling techniques based on predicting the issue cycle of an instruction [1,8,9,16,19,24,26,36] remove the wakeup delay from the critical path and remove the CAM logic from instruction wakeup, but need to keep track of the cycle when each physical register will become ready. In [13], the wakeup time prediction occurs in parallel with the instruction fetching.…”