Using utility prediction models to dynamically choose program thread counts

Abstract

Multithreaded applications can simultaneously execute on a chip multiprocessor computer, starting and stopping without warning or pattern. The behavior of each program can be different, interacting in unexpected ways, including causing competition for CPU cycles, which harms performance.To maximize program performance in this type of dynamic execution environment, the interactions among applications must be controlled. These interactions can be controlled by carefully choosing the number of threads for each multithreaded application (i.e., a system configuration). To choose a configuration, we advocate using program utility models to predict application behavior. Only a system that is capable of predicting and analyzing performance under multiple configurations can choose the best configuration and thus robustly meet its performance goals. In this paper, we present such a system.Our approach first gathers profile data. The profile data is used by multiple linear regression to build a utility model. The model takes into account program scalability, susceptibility to interference, and any inherent leveling off of performance as a program's thread count is increased. A utility model is constructed for each application. When the system workload changes, the utility models are consulted to find the new configuration that maximizes system performance while meeting each program's quality of service goals. We use multithreaded applications from PARSEC to evaluate our approach. Compared to the best traditional policy, which does not consider variances in the dynamic workload, our approach simultaneously improves system throughput by 19.3% while meeting user performance constraints 28% more often.

Reconfigurable multi-core architecture - a plausible solution to the von Neumann performance bottleneck

“…This is also known as auto-tuning, which decides the granularity of a program running on multi-core processors. There are several work (Moore and Childers, 2012;Kurzak et al, 2012;Sato et al, 2011;Vuduc and Moon, 2005) discussing about auto-tuning of applications on multi-core processors. The previous work focus on explicitly searching the partially-pruned parameter space.…”

RCS

“…Although we share a similar mechanism to dynamically change the number of operating cores at runtime, they fixed the number of cores in their study. Moore and Childers also proposed a dynamic approach that can choose the optimal number of threads based on offline-and online-methods [17]. Finally, some prior studies investigated adaptive approaches based on ML, control theory, etc.…”

Adapt

“…In Moore and Childers [2012] statistical models are used to predict appropriate number of threads for a multithreaded program when it is running with another multithreaded program on a multicore system for achieving high throughput. It is shown that by varying the number of threads of an application, according to the workload, improvements in throughput can be obtained.…”

“…Moreover, unlike the above work, ADAPT aims to achieve both low total turnaround times and high system utilization. Most importantly, Moore and Childers [2012] do not exploit application characteristics such as lock contention, scheduling policies, and memory allocation policies that ADAPT exploits. Allocating appropriate number of cores is also a critical factor in coscheduling multithreaded programs that ADAPT considers.…”

“…Allocating appropriate number of cores is also a critical factor in coscheduling multithreaded programs that ADAPT considers. One can also view the techniques of Moore and Childers [2012] and ADAPT as complementary techniques that can be potentially integrated to further improve performance.…”