Software prefetching for indirect memory accesses

Ainsworth, Sam; Jones, Timothy M.

doi:10.1109/cgo.2017.7863749

Cited by 42 publications

(54 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Software Prefetching: Software prefetching [28][29][30][31][32][33] provides a way for programmers to insert prefetching instructions into a program targeting various simple and complex patterns. In Ainsworth [34], while the insertion of software prefetches for indirect memory accesses is automated and eliminates the requirement for programmer effort, it cannot guarantee insertion of the instructions in an optimized way for a specific architecture. Furthermore, significant instruction overhead may offset its benefits.…”

Section: Related Workmentioning

confidence: 99%

Fast Key-Value Lookups with Node Tracker

Cavus

Shatnawi

Sendag

et al. 2021

ACM Trans. Archit. Code Optim.

View full text Add to dashboard Cite

Lookup operations for in-memory databases are heavily memory bound, because they often rely on pointer-chasing linked data structure traversals. They also have many branches that are hard-to-predict due to random key lookups. In this study, we show that although cache misses are the primary bottleneck for these applications, without a method for eliminating the branch mispredictions only a small fraction of the performance benefit is achieved through prefetching alone. We propose the Node Tracker (NT), a novel programmable prefetcher/pre-execution unit that is highly effective in exploiting inter key-lookup parallelism to improve single-thread performance. We extend NT with branch outcome streaming (BOS) to reduce branch mispredictions and show that this achieves an extra 3× speedup. Finally, we evaluate the NT as a pre-execution unit and demonstrate that we can further improve the performance in both single- and multi-threaded execution modes. Our results show that, on average, NT improves single-thread performance by 4.1× when used as a prefetcher; 11.9× as a prefetcher with BOS; 14.9× as a pre-execution unit and 18.8× as a pre-execution unit with BOS. Finally, with 24 cores of the latter version, we achieve a speedup of 203× and 11× over the single-core and 24-core baselines, respectively.

show abstract

Section: Related Workmentioning

confidence: 99%

Fast Key-Value Lookups with Node Tracker

Cavus

Shatnawi

Sendag

et al. 2021

ACM Trans. Archit. Code Optim.

View full text Add to dashboard Cite

show abstract

“…In contrast, SWOOP reuses previously computed values and loaded data to reduce overhead and its execution is triggered and hidden by hardware stalls that would be pure performance loss otherwise. Software prefetching [3,21,39,73] lacks precision. Code containing complex control flow cannot rely on softwareprefetch instructions inserted a few iterations ahead.…”

Section: Related Workmentioning

confidence: 99%

SWOOP: software-hardware co-design for non-speculative, execute-ahead, in-order cores

et al. 2018

View full text Add to dashboard Cite

Increasing demands for energy efficiency constrain emerging hardware. These new hardware trends challenge the established assumptions in code generation and force us to rethink existing software optimization techniques. We propose a cross-layer redesign of the way compilers and the underlying microarchitecture are built and interact, to achieve both performance and high energy efficiency.In this paper, we address one of the main performance bottlenecks-last-level cache misses-through a softwarehardware co-design. Our approach is able to hide memory latency and attain increased memory and instruction level parallelism by orchestrating a non-speculative, execute-ahead paradigm in software (SWOOP). While out-of-order (OoO) architectures attempt to hide memory latency by dynamically reordering instructions, they do so through expensive, power-hungry, speculative mechanisms. We aim to shift this complexity into software, and we build upon compilation techniques inherited from VLIW, software pipelining, modulo scheduling, decoupled access-execution, and software prefetching. In contrast to previous approaches we do not rely on either software or hardware speculation that can be detrimental to efficiency. Our SWOOP compiler is enhanced with lightweight architectural support, thus being able to transform applications that include highly complex control-flow and indirect memory accesses.The effectiveness of our software-hardware co-design is proven on the most limited but energy-efficient microarchitectures, non-speculative, in-order execution (InO) cores, which rely entirely on compile-time instruction scheduling.

show abstract

“…In contrast, SWOOP reuses previously computed values and loaded data to reduce overhead and its execution is triggered and hidden by hardware stalls that would be pure performance loss otherwise. Software prefetching [3,21,39,73] lacks precision. Code containing complex control low cannot rely on softwareprefetch instructions inserted a few iterations ahead.…”

Section: Related Workmentioning

confidence: 99%

SWOOP: software-hardware co-design for non-speculative, execute-ahead, in-order cores

Tran

Jimborean

Carlson

et al. 2018

Proceedings of the 39th ACM SIGPLAN Conference on Programming Language Design and Implementation

View full text Add to dashboard Cite

Increasing demands for energy eiciency constrain emerging hardware. These new hardware trends challenge the established assumptions in code generation and force us to rethink existing software optimization techniques. We propose a cross-layer redesign of the way compilers and the underlying microarchitecture are built and interact, to achieve both performance and high energy eiciency. In this paper, we address one of the main performance bottlenecksÐlast-level cache missesÐthrough a softwarehardware co-design. Our approach is able to hide memory latency and attain increased memory and instruction level parallelism by orchestrating a non-speculative, execute-ahead paradigm in software (SWOOP). While out-of-order (OoO) architectures attempt to hide memory latency by dynamically reordering instructions, they do so through expensive, power-hungry, speculative mechanisms. We aim to shift this complexity into software, and we build upon compilation techniques inherited from VLIW, software pipelining, modulo scheduling, decoupled access-execution, and software prefetching. In contrast to previous approaches we do not rely on either software or hardware speculation that can be detrimental to eiciency. Our SWOOP compiler is enhanced with lightweight architectural support, thus being able to transform applications that include highly complex control-low and indirect memory accesses.

show abstract

Software prefetching for indirect memory accesses

Cited by 42 publications

References 19 publications

Fast Key-Value Lookups with Node Tracker

Fast Key-Value Lookups with Node Tracker

SWOOP: software-hardware co-design for non-speculative, execute-ahead, in-order cores

SWOOP: software-hardware co-design for non-speculative, execute-ahead, in-order cores

Contact Info

Product

Resources

About