Predicting the Flexibility of Dynamic Loop Scheduling Using an Artificial Neural Network

Srivastava, Srishti; Malone, Brandon; Sukhija, Nitin; Banicescu, Ioana; Ciorba, Florina M.

doi:10.1109/ispdc.2013.10

Cited by 9 publications

(17 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…These clusters are evaluated based on the responsiveness of programs within each cluster to optimizations. Srivastava et al [15] use a neural network to predict how changing loop scheduling policy affects load balancing in an already parallelized application. Although not directly predicting parallelization, Park et al [11] use an SVM with a kernel function operating on the static control flow graph of a program to predict the improvements gained from compiler optimizations, and demonstrate that this graph-based classification improves performance.…”

Section: Related Workmentioning

confidence: 99%

Predicting Parallelization of Sequential Programs Using Supervised Learning

Fried

Jannesari

et al. 2013

2013 12th International Conference on Machine Learning and Applications

View full text Add to dashboard Cite

We investigate an automatic method for classifying which regions of sequential programs could be parallelized, using dynamic features of the code collected at runtime. We train a supervised learning algorithm on versions of the NAS Parallel Benchmark (NPB) [14] code hand-annotated with OpenMP parallelization directives in order to approximate the parallelization that might be produced by a human expert. A model comparison shows that support vector machines and decision trees have comparable performance on this classification problem, but boosting using AdaBoost is able to increase the performance of the decision trees. We further analyze the relative importance of the collected program features and demonstrate that withinloop instruction counts provide the greatest contribution to decision tree error reduction, with dependency graph features of secondary importance.

show abstract

Section: Related Workmentioning

confidence: 99%

Predicting Parallelization of Sequential Programs Using Supervised Learning

Fried

Jannesari

et al. 2013

2013 12th International Conference on Machine Learning and Applications

View full text Add to dashboard Cite

show abstract

“…Significantly extending previous work [9], herein we formally define empirical robustness prediction models, which predict the robustness of a DLS algorithm on a given instance. This is a regression problem; consequently, we explore the vast space of regression model classes, their hyperparameters and parameters which solve the regression problem, and select the model which best predicts the robustness of a scheduling algorithm on any instance.…”

Section: Introductionmentioning

confidence: 95%

“…In recent work [9], a proof of concept engaging a multilayer perceptron (MLP) artificial neural network (ANN) was presented for predicting the flexibility of individual DLS algorithms in parallel and distributed environments. However, the problem of deciding at runtime which DLS algorithm to use for a given instance was not addressed.…”

Section: Introductionmentioning

confidence: 99%

“…The major contributions of this paper are three-fold: (i) Employing state-of-the-art machine learning techniques to explore a large space of empirical robustness prediction models. Consequently, the learned models predict robustness much more accurately than the previous models [9]; (ii) Selecting DLS algorithms from a portfolio with the developed empirical robustness prediction models. The prediction models enable an algorithm selection on a per-instance basis; (iii) Evaluating experimentally the performance of the perinstance selections compared to the simpler winner-take-all approach.…”

Section: Introductionmentioning

confidence: 99%

“…Ei ← E * 8: end for 9: return E In our previous work, we trained a multilayer perceptron with hand-tuned hyperparameters to serve as an empirical robustness prediction model [9]. Our new workflow (shown in Figure 1) adopts a much more general strategy: we run Auto-WEKA on the training set to search both for good model classes and hyperparameters.…”

mentioning

confidence: 99%

See 2 more Smart Citations

Portfolio-Based Selection of Robust Dynamic Loop Scheduling Algorithms Using Machine Learning

Sukhija

Malone

Srivastava

et al. 2014

2014 IEEE International Parallel &Amp; Distributed Processing Symposium Workshops

Self Cite

View full text Add to dashboard Cite

The execution of computationally intensive parallel applications in heterogeneous environments, where the quality and quantity of computing resources available to a single user continuously change, often leads to irregular behavior, in general due to variations of algorithmic and systemic nature. To improve the performance of scientific applications, loop scheduling algorithms are often employed for load balancing of their parallel loops. However, it is a challenge to select the most robust scheduling algorithms for guaranteeing optimized performance of scientific applications on large-scale computing systems that comprise resources which are widely distributed, highly heterogeneous, often shared among multiple users, and have computing availabilities that cannot always be guaranteed or predicted. To address this challenge, in this work we focus on a portfolio-based approach to enable the dynamic selection and use of the most robust dynamic loop scheduling (DLS) algorithm from a portfolio of DLS algorithms, depending on the given application and current system characteristics including workload conditions. Thus, in this paper we provide a solution to the algorithm selection problem and experimentally evaluate its quality. We propose the use of supervised machine learning techniques to build empirical robustness prediction models that are used to predict DLS algorithm's robustness for given scientific application characteristics and system availabilities. Using simulated scientific applications characteristics and system availabilities, along with empirical robustness prediction models, we show that the proposed portfolio-based approach enables the selection of the most robust DLS algorithm that satisfies a user-specified tolerance on the given application's performance obtained in the particular computing system with a certain variable availability. We also show that the portfoliobased approach offers higher guarantees regarding the robust performance of the application using the automatically selected DLS algorithms when compared to the robust performance of the same application using a manually selected DLS algorithm.

show abstract

Design methodology for workload‐aware loop scheduling strategies based on genetic algorithm and simulation

Penna

Castro

Freitas

et al. 2016

Concurrency and Computation

View full text Add to dashboard Cite

International audienceIn high-performance computing, the application's workload must be evenly balanced among threads to deliver cutting-edge performance and scalability. In OpenMP, the load balancing problem arises when scheduling loop iterations to threads. In this context, several scheduling strategies have been proposed, but they do not take into account the input workload of the application and thus turn out to be suboptimal. In this work, we introduce a design methodology to propose, study, and assess the performance of workload-aware loop scheduling strategies. In this methodology, a genetic algorithm is employed to explore the state space solution of the problem itself and to guide the design of new loop scheduling strategies, and a simulator is used to evaluate their performance. As a proof of concept, we show how the proposed methodology was used to propose and study a new workload-aware loop scheduling strategy named smart round-robin (SRR). We implemented this strategy into GNU Compiler Collection's OpenMP runtime. We carry out several experiments to validate the simulator and to evaluate the performance of SRR. Our experimental results show that SRR may deliver up to 37.89% and 14.10% better performance than OpenMP's dynamic loop scheduling strategy in the simulated environment and in a real-world application kernel, respectively

show abstract

Predicting the Flexibility of Dynamic Loop Scheduling Using an Artificial Neural Network

Cited by 9 publications

References 14 publications

Predicting Parallelization of Sequential Programs Using Supervised Learning

Predicting Parallelization of Sequential Programs Using Supervised Learning

Portfolio-Based Selection of Robust Dynamic Loop Scheduling Algorithms Using Machine Learning

Design methodology for workload‐aware loop scheduling strategies based on genetic algorithm and simulation

Contact Info

Product

Resources

About