Adaptive prefetching for shared cache based chip multiprocessors

Abstract

Chip multiprocessors (CMPs) present a unique scenario for software data prefetching with subtle tradeoffs between memory bandwidth and performance. In a shared L2 based CMP, multiple cores compete for the shared on-chip cache space and limited off-chip pin bandwidth. Purely software based prefetching techniques tend to increase this contention, leading to degradation in performance. In some cases, prefetches can become harmful by kicking out useful data from the shared cache whose next usage is earlier than the prefetched data, and the fraction of such harmful prefetches usually increases when we increase the number of cores used for executing a multi-threaded application code. In this paper, we propose two complementary techniques to address the problem of harmful prefetches in the context of shared L2 based CMPs. These techniques, namely, suppressing select data prefetches (if they are found to be harmful) and pinning select data in the L2 cache (if they are found to be frequent victim of harmful prefetches), are evaluated in this paper using two embedded application codes. Our experiments demonstrate that these two techniques are very effective in mitigating the impact of harmful prefetches, and as a result, we extract significant benefits from software prefetching even with large core counts.

Collective Memory Transfers for Multi-Core Chips

“…Unpredictable memory access streams make extracting benefits from memory prefetching difficult [37,57]. Depending on the access pattern, only 14%-97% of memory bandwidth can actually be utilized [57].…”

“…This is done with local independent hardware prefetch [37] or cache fill streams for a cache-coherent CMP, a list of outstanding load-store requests for a massively multithreaded architecture like a GPU, or via a sequence of DMA requests for a local store architecture like STI Cell [62]. In all of these cases, requests are sent independently over an unpredictable network and thus arrive in nearly random order to memory [76,69].…”

“…Memory prefetching techniques focus on reducing latency and offer little benefit in systems that are bound by memory bandwidth. Prefetching techniques typically perform predictions independently at each processor and thus create out-of-order access patterns [37].…”

Collective memory transfers for multi-core chips

“…This is done with local independent hardware prefetch [19] or cache fill streams for a cache-coherent CMP, a list of outstanding load-store requests for a massively multithreaded architecture like a GPU, or via a sequence of DMA requests for a local store architecture like STI Cell [34]. In all of these cases, requests are sent independently over an unpredictable network and thus arrive in nearly random order to memory [38,42].…”

“…Memory prefetching techniques focus on reducing latency and offer little benefit in systems that are bound by memory bandwidth. Prefetching techniques typically perform predictions independently at each processor and thus create out-oforder access patterns [19].…”