Re-establishing Fetch-Directed Instruction Prefetching: An Industry Perspective

Ishíi, Yasuo; Lee, Jaekyu; Nathella, Krishnendra; Sunwoo, Dam

doi:10.1109/ispass51385.2021.00034

Cited by 22 publications

(19 citation statements)

References 27 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Simulation parameters. We simulate and evaluate Thermometer using the ChampSim [5] simulator and adjust simulation parameters to resemble a recent state-of-the-art industry FDIP baseline [57,58], as listed in Table 1. We implement the optimal BTB replacement policy (Belady's algorithm [29,61]) and other existing policies including SRRIP [62], GHRP [20], and Hawkeye [60] to compare them with Thermometer.…”

Section: Experimental Methodologymentioning

confidence: 99%

“…Using these traces, we characterize BTB replacement challenges to design Thermometer, a novel profileguided BTB replacement technique. We validate Thermometer's effectiveness on 13 data center applications and on CBP-5 [15] and IPC-1 [17] traces that prior work [20,24,57,58] evaluate their frontend optimizations.…”

Section: Experimental Methodologymentioning

confidence: 99%

“…FDIP decouples the branch prediction unit from the instruction fetch unit so that the frontend can run ahead, producing the instruction addresses likely to be executed in the near future. Prefetching I-cache lines corresponding to these future accesses avoids potential frontend stalls [83,84], providing performance similar to aggressive I-cache prefetchers [57,58].…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Thermometer

Song

Khan

Shahri

et al. 2022

Proceedings of the 49th Annual International Symposium on Computer Architecture

View full text Add to dashboard Cite

Modern processors employ a decoupled frontend with Fetch Directed Instruction Prefetching (FDIP) to avoid frontend stalls in data center applications. However, the large branch footprint of data center applications precipitates frequent Branch Target Buffer (BTB) misses that prohibit FDIP from eliminating more than 40% of all frontend stalls. We find that the state-of-the-art BTB optimization techniques (e.g., BTB prefetching and replacement mechanisms) cannot eliminate these misses due to their inadequate understanding of branch reuse behavior in data center applications.In this paper, we first perform a comprehensive characterization of the branch behavior of data center applications, and determine that identifying optimal BTB replacement decisions requires considering both transient and holistic (i.e., across the entire execution) branch behavior. We then present Thermometer, a novel BTB replacement technique that realizes the holistic branch behavior via a profile-guided analysis. Based on the collected profile, Thermometer generates useful BTB replacement hints that the underlying hardware can leverage. We evaluate Thermometer using 13 widelyused data center applications and demonstrate that it provides an average speedup of 8.7% (0.4%-64.9%) while outperforming the state-of-the-art BTB replacement techniques by 5.6× (on average, the best performing prior work achieves 1.5% speedup). We also demonstrate that Thermometer achieves a performance speedup that is, on average, 83.6% of the speedup achieved by the optimal BTB replacement policy.

show abstract

Section: Experimental Methodologymentioning

confidence: 99%

Section: Experimental Methodologymentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Thermometer

Song

Khan

Shahri

et al. 2022

Proceedings of the 49th Annual International Symposium on Computer Architecture

View full text Add to dashboard Cite

show abstract

“…The request for cache block containing these instructions is sent to the L1I, effectively hiding the latency for future instructions as multiple L1I requests are going in parallel. Recent work [22] also shows that L1I prefetchers like EIP [23] and FNL+MMA [24] do not provide significant performance improvement with a decoupled front-end and FDIP prefetcher.…”

Section: Motivationmentioning

confidence: 99%

“…A recent work [22] has shown that with a decoupled frontend, an FDIP prefetcher, and the ideal BTB, the ideal L1I does not provide a significant performance improvement. We also observe the same in Figure 1 where an ideal front-end provides 3% more improvement compared to the ideal BTB.…”

Section: Introductionmentioning

confidence: 99%

Micro BTB: A High Performance and Lightweight Last-Level Branch Target Buffer for Servers

Gupta¹,

Panda²

2021

Preprint

View full text Add to dashboard Cite

High-performance branch target buffers (BTBs) and the L1I cache are key to high-performance front-end. Modern branch predictors are highly accurate, but with an increase in code footprint in modern-day server workloads, BTB and L1I misses are still frequent. Recent industry trend shows usage of large BTBs (100s of KB per core) that provide performance closer to the ideal BTB along with a decoupled front-end that provides efficient fetch-directed L1I instruction prefetching. On the other hand, techniques proposed by academia, like BTB prefetching and using retire order stream for learning, fail to provide significant performance with modern-day processor cores that are deeper and wider.We solve the problem fundamentally by increasing the storage density of the last-level BTB. We observe that not all branch instructions require a full branch target address. Instead, we can store the branch target as a branch offset, relative to the branch instruction. Using branch offset enables the BTB to store multiple branches per entry. We reduce the BTB storage in half, but we observe that it increases skewness in the BTB. We propose a skewed indexed and compressed last-level BTB design called MicroBTB (MBTB) that stores multiple branches per BTB entry. We evaluate MBTB on 100 industry-provided server workloads. A 4K-entry MBTB provides 17.61% performance improvement compared to an 8K-entry baseline BTB design with a storage savings of 47.5KB per core.

show abstract