Thread Assignment in Multicore/Multithreaded Processors: A Statistical Approach

Abstract

Abstract-The introduction of multicore/multithreaded processors, comprised of a large number of hardware contexts (virtual CPUs) that share resources at multiple levels, has made process scheduling, in particular assignment of running threads to available hardware contexts, an important aspect of system performance. Nevertheless, thread assignment of applications running on state-of-the art processors is an NP-complete problem. Over the years, numerous studies have proposed heuristic-based algorithms for thread assignment. Since the thread assignment problem is intractable, it is in general impossible to know the performance of the optimal assignment, so the room for improvement of a given algorithm is also unknown. It is therefore hard to decide whether to invest more effort and time to improve an algorithm that may already be close to optimal. In this paper, we present a statistical approach to the thread assignment problem. First, we present a method that predicts the performance of the optimal thread assignment, based on the observed performance of each thread assignment in a random sample. The method is based on Extreme Value Theory (EVT), a branch of statistics that analyses extreme deviations from the population mean. We also propose sample pruning, a method that significantly reduces the time required to apply the statistical method by reducing the number of candidate solutions that need to be measured. Finally, we show that, if no suitable heuristicbased algorithm is available, a sample of several thousand random thread assignments is enough to obtain, with high confidence, an assignment with performance close to optimal. The presented approach is architecture and application independent, and it can be used to address the thread assignment problem in various domains. It is especially well suited for systems in which the workload seldom changes. An example is network systems, which typically provide a constant set of services that are known in advance, with network applications performing a similar processing algorithm for each packet in the system. In this paper, we validate our methods with an industrial case study for a set of multithreaded network applications on an UltraSPARC T2 processor. This article is an extension of our previous work [44], which was published in

Efficient thread/page/parallelism autotuning for NUMA systems

“…Each of the previous works is able to achieve the best performance for some of the benchmarks, but none of them explores enough of the search space to achieve the best for all, as we do. (PPP=Page Placement Policy, PND=Page NUMA Degree, TPP=Thread Placement Policy, NT=Number of Threads, TND=Thread NUMA Degree) Previous work PPP: [6,10,23,24,26], TPP/NT/TND: [12,17,27,29,30,34], PPP/TPP: [11]. Our work includes all optimizations and performs significantly better.…”

“…For cg and needle, one region in each (needle_176 and cg_405) can obtain the optimal performance with PPP/PND, but their other regions (needle_116 and cg_551) require the full search space to achieve the best results. Overall, our full search is able to achieve an average speedup of 2.48× (median 2.23×) compared to 1.84× (median 1.27×) for the best subset of existing TPP/NT/TND searches [12,17,27,29,30,34]. Figure 8 (top) shows the predicted speedups from optimizing the first instance of each region in the application via the codelet vs. the actual performance achieved by all instances of that region in the application with the optimization.…”

Upper Bounds Prediction of the Execution Time of Programs Running on ARM Cortex-A Systems

“…An approach based on EVT theory has also been used for optimal performance analysis. Radojković et al [5] presented a new method for predicting the performance of the thread assignment in multi-core processors. Using statistical inference of each thread assignment in a random sample, the authors estimates the optimal one.…”

“…In the related work, a number of methods for application of EVT theory for WCET estimations has been proposed [2], [5], [6]; nevertheless, a considerable uncertainty due to the complexity of the problem still exists. Results of statistical tests are often fuzzy and it is hard to make a correct decision on their basis to fulfill EVT requirements.…”

Task-Aware Priority Scheduling for Multicore Processors

“…when BlackBox makes scheduling decision, it considers two tables to make performance prediction and picks out the best solution. In new research [7], Radojkovicp presents the best scheduling scheme by combining Extreme Value theory and performance evaluation of historical scheduling result.…”