Helper threads via virtual multithreading on an experimental itanium ^® 2 processor-based platform

Abstract

Helper threading is a technology to accelerate a program by exploiting a processor's multithreading capability to run ``assist'' threads. Previous experiments on hyper-threaded processors have demonstrated significant speedups by using helper threads to prefetch hard-to-predict delinquent data accesses. In order to apply this technique to processors that do not have built-in hardware support for multithreading, we introduce virtual multithreading (VMT), a novel form of switch-on-event user-level multithreading, capable of fly-weight multiplexing of event-driven thread executions on a single processor without additional operating system support. The compiler plays a key role in minimizing synchronization cost by judiciously partitioning register usage among the user-level threads. The VMT approach makes it possible to launch dynamic helper thread instances in response to long-latency cache miss events, and to run helper threads in the shadow of cache misses when the main thread would be otherwise stalled.The concept of VMT is prototyped on an Itanium ® 2 processor using features provided by the Processor Abstraction Layer (PAL) firmware mechanism already present in currently shipping processors. On a 4-way MP physical system equipped with VMT-enabled Itanium 2 processors, helper threading via the VMT mechanism can achieve significant performance gains for a diverse set of real-world workloads, ranging from single-threaded workstation benchmarks to heavily multithreaded large scale decision support systems (DSS) using the IBM DB2 Universal Database. We measure a wall-clock speedup of 5.8% to 38.5% for the workstation benchmarks, and 5.0% to 12.7% on various queries in the DSS workload.

Pythia: A Customizable Hardware Prefetching Framework Using Online Reinforcement Learning

“…We divide the traditional prefetching algorithms into three broad categories: precomputation-based, temporal, and spatial. Precomputation-based prefetchers (e.g., runahead [46,59,60,63,[95][96][97][98][99][100][101] and helper-thread execution [33,40,41,45,73,74,87,106,119,128,141,144]) pre-execute program code to generate future memory requests. These prefetchers can generate highly-accurate prefetches even when no recognizable pattern exists in program memory requests.…”

Helper Without Threads: Customized Prefetching for Delinquent Irregular Loads

“…We call such DILs prefetchable and generate customized prefetching code for each. Similar to helper threads [34], [68], the prefetching code is extracted from the dataflow of the application itself. However, unlike helper threads (which require either spare thread contexts or special hardware support to run), our method inserts the prefetcher code into the application machine code directly.…”

“…Helper threads [12], [13], [15], [16], [20], [21], [31], [32], [34], [35], [41], [42], [43], [59], [67], [68], [75] extract the backward slice of a delinquent load and run it on a spare thread context. When the latency of the backward slice is less than that of the original loop, the helper thread runs ahead of the main thread and prefetches memory accessed by the main thread into the cache.…”

“…When the latency of the backward slice is less than that of the original loop, the helper thread runs ahead of the main thread and prefetches memory accessed by the main thread into the cache. This technique has the advantage of being flexible enough to be implemented in hardware [12], [13], [15], [16], [20], [21], [23], [43], [59], [67], [75], or software [31], [32], [34], [35], [41], [42], [68]. It can also work in the absence of high-level source code and has been demonstrated in a compiler [35], binary tool [41], or a dynamic optimizer [42].…”

“…Speculative precomputation [12], [13], [15], [16], [20], [21], [43], [59], [67], [75], otherwise known as helper threading [31], [32], [34], [35], [41], [42], [68] is such a technique. It reduces the single-thread latency of an application by using idle thread contexts in the hardware to spawn special-purpose, speculative threads called helper threads.…”