The Performance Optimization of Threaded Prefetching for Linked Data Structures

Abstract

Helper threaded prefetching based on Chip Multiprocessor is a well known approach to reducing memory latency and has been explored in linked data structures accesses. However, conventional helper threaded prefetching often suffers from useless prefetches and cache thrashing, which affect its effectiveness. In this paper, we first analyzed the shortcomings of conventional helper threaded prefetching for linked data structures. Then we proposed an improved helper threaded prefetching, Skip Helper Threaded Prefetching, for hotspots with two level data traversals. Our solution is to profile the applications and balance delinquent loads between main thread and prefetching thread based on the characteristic of operations in their hotspots. Evaluations show that the proposed solution improves average performance by 8.9% (-O2) and 8.5% (-O3) over the conventional helper threaded prefetching that greedily prefetches all delinquent loads. We also compare our proposal with the active threaded prefetching which synchronizes with main thread by semaphore, and find that our proposal provides better performance for the targeted applications.

Estimating Effective Prefetch Distance in Threaded Prefetching for Linked Data Structures

“…1a shows that a majority of last-level cache misses are caused by the second level traversals in inner for-loop. As indicated in our earlier works [20,28], for LDS programs with low CALR, the helper thread may be too stressed to run ahead of the main thread, thus making no performance gain. Nonetheless, this problem can be solved if the helper thread ignores some problem loads.…”

“…The selection of proper R P has been illustrated in our earlier work [28]. The right R P can improve the parallelism of helper thread and main thread.…”

“…In this section, we briefly describe skip helper threaded prefetching technique, which proposed in our earlier work [28] and our proposal builds upon.…”

“…In our earlier work [28], Skip Helper Threaded Prefetching technique was proposed to improve the effectiveness of helper thread. Like conventional helper threaded prefetching mechanism [10,21,22,26], our SHTP mechanism focuses on loops with heavy cache misses.…”

“…In our earlier work [28], we proposed an improved threaded prefetching technique, skip helper threaded prefetching (SHTP), to overcome these problems. SHTP staggers memory accesses between main thread and helper thread based on the characteristic of operations in hotspots.…”