Efficient thread/page/parallelism autotuning for NUMA systems

Abstract

Current multi-socket systems have complex memory hierarchies with significant Non-Uniform Memory Access (NUMA) effects: memory performance depends on the location of the data and the thread. This complexity means that thread-and data-mappings have a significant impact on performance. However, it is hard to find efficient data mappings and thread configurations due to the complex interactions between applications and systems. In this paper we explore the combined search space of thread mappings, data mappings, number of NUMA nodes, and degreeof-parallelism, per application phase, and across multiple systems. We show that there are significant performance benefits from optimizing this wide range of parameters together. However, such an optimization presents two challenges: accurately modeling the performance impact of configurations across applications and systems, and exploring the vast space of configurations. To overcome the modeling challenge, we use native execution of small, representative codelets, which reproduce the system and application interactions. To make the search practical, we build a search space by combining a range of state of the art thread-and data-mapping policies. Combining these two approaches results in a tractable search space that can be quickly and accurately evaluated without sacrificing significant performance. This search finds non-intuitive configurations that perform significantly better than previous works. With this approach we are able to achieve an average speedup of 1.97× on a four node NUMA system.

Modeling and optimizing NUMA effects and prefetching with machine learning

“…Codelet execution extracts hot regions from the application as small, representative codelets and uses them to characterize the application's performance. Codelets are on average 66× faster [35] to evaluate than running the full application. Codelet execution is faster because it only executes a few instances of each region (instead of hundreds during the original run).…”

“…Codelets have been shown to be quite accurate for both microarchitectural evaluation [12] and NUMA configuration studies [35]. This is because parallel regions typically exhibit similar behavior [40].…”

“…Such inter-region configuration conflicts have been shown to have a small impact on overall performance 10 . Popov et al [35] reported that only a few applications in the NAS and Rodinia benchmarks, such as BT ans SP, are significantly affected, and the overall reduction in speedup when accounting for inter-region conflicts was only 6%. We therefore evaluate the impact of inter-region conflicts on our online profiling approach for both BT and SP.…”

“…Our approach is similar to the one proposed by Popov et al [35]. First, we select the best configuration for each region individually.…”

“…Exploration strategies have evaluated different thread mappings [36,37] and degrees of parallelism [17]. CERE [33,35] used codelets to more rapidly evaluate configurations, allowing the simultaneous exploration of thread and page mappings for further benefit.…”