EXTES: An Execution-Time Estimation Scheme for Efficient Computational Science and Engineering Simulation via Machine Learning

Abstract

In recent years, computational science and engineering (CSE) simulations using high-performance computing resources are actively exploited to solve complex domain-specific problems. Thanks to the remarkable advance of IT technology, the CSE community is challenging more complex and difficult problems than ever before, by running these simulations online. In this regard, we often witness that 1) online simulation users suffer from knowing little about the estimated termination time of their launched simulations and 2) the limited computing resources are squandered by wrong input that leads the simulations to run forever. To address such issues, we propose a novel execution time estimation scheme, termed EXTES, using machine learning techniques for more efficient online CSE simulations. With a large amount of existing provenance data, the EXTES scheme trains a suite of models rooted from classification, regression, and a hybrid of the two and utilize these models to estimate the execution time for specified input parameters for simulations. In the experiments on real simulation data, our proposed models achieved about 73% accuracy on average in execution time estimation across 16 simulation programs taken from a variety of CSE fields. In the meantime, the overhead incurred by the training and estimation is almost negligible.

CLUTCH: A Clustering-Driven Runtime Estimation Scheme for Scientific Simulations

“…Their source code is not open to public. Some other work, however, was reproducible; for instance, Tyryshkina et al's work [17] and our earlier work [18] made their code available, and Pumma's algorithm [16] was easily reimplemented. In Section VII, we will compare the performance of our proposed scheme with that of these three previous methods on the same simulation datasets.…”

“…Second, the target of estimation is slightly different across the studies we have investigated. Most of the existing works [2], [9]- [18], [29], [30] aim to predict the runtime. Some studies [2], [8] attempt to estimate the memory usage.…”

“…Fourth, the availability of hardware-specific parameter data differs from the existing works. The algorithms proposed in some of the studies [2], [12]- [14], [16], [17], [24], [29] require hardware information that may enhance the quality of the runtime estimation, while those proposed in some other studies [15], [18] require no such information.…”

“…Last but not least, the previous run-time estimation method [18], named EXTES, is an ensemble of classification and regression. Here are the major distinctions between EXTES, and the proposed scheme, CLUTCH.…”

“…In this paper we present a novel "ensemble" of clustering, regression, and classification to construct a runtime estimation model. Unlike previous works [13], [18] which take a similar approach, the biggest difference in our work is to apply clustering to the input data in the early training process, resulting in estimation quality enhancement.…”