Trade-Off Between Prediction Accuracy and Underestimation Rate in Job Runtime Estimates

Fan, Yuping; Rich, Paul M.; Allcock, William E.; Papka, Michael E.; Lan, Zhiling

doi:10.1109/cluster.2017.11

Cited by 46 publications

(23 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Rodrigues et al [6] predicted job execution, wait time, and memory usage with job logs and batch schedulers by an ensemble of machine learning algorithms such as RF and kNN. Fan et al [7] proposed an online runtime adjustment framework for trade-off between prediction accuracy and underestimation rate in job runtime estimates.…”

Section: Related Workmentioning

confidence: 99%

Machine Learning Predictions for Underestimation of Job Runtime on HPC System

Guo

Nomura

Barton

et al. 2018

Lecture Notes in Computer Science

View full text Add to dashboard Cite

In modern high-performance computing (HPC) systems, users are usually requested to estimate the job runtime for system scheduling when they submit a job. In general, an underestimation of job runtime will cause the HPC system to terminate the job before its completion. If users could be notified that their jobs may not finish before its allocated time expires, users can take actions, such as killing the job and resubmitting it after parameter adjustment, to save time and cost. Meanwhile, the productivity of HPC systems could also be vastly improved. In this paper, we propose a data-driven approach -that is, one that actively observes, analyzes, and logs jobs -for predicting underestimation of job runtime on HPC systems. Using data produced by TSUBAME 2.5, a supercomputer deployed at the Tokyo Institute of Technology, we apply machine learning algorithms to recognize patterns about whether the underestimation of job runtime occurs. Our experimental results show that our approach on runtime-underestimation prediction with 80% precision, 70% recall and 74% F1-score on the entirety of a given dataset. Finally, we split the entire job data set into subsets categorized by scientific application name. The best precision, recall and F1-score of subsets on runtime-underestimation prediction achieved 90%, 95% and 92% respectively.

show abstract

Section: Related Workmentioning

confidence: 99%

Machine Learning Predictions for Underestimation of Job Runtime on HPC System

Guo

Nomura

Barton

et al. 2018

Lecture Notes in Computer Science

View full text Add to dashboard Cite

show abstract

“…In [10], Fan Y et al mainly started from the point of view of data processing, hoping to optimize and filter data to improve job running time prediction accuracy. A lot of work before this has been devoted to improving the prediction accuracy by reducing the overestimation of running time but did not solve the problem of underestimation.…”

Section: Related Workmentioning

confidence: 99%

Predicting running time of aerodynamic jobs in HPC system by combining supervised and unsupervised learning method

Wang

Dai

et al. 2021

Adv. Aerodyn.

View full text Add to dashboard Cite

Improving resource utilization is an important goal of high-performance computing systems of supercomputing centers. To meet this goal, the job scheduler of high-performance computing systems often uses backfilling scheduling to fill short-time jobs into job gaps at the front of the queue. Backfilling scheduling needs to obtain the running time of the job. In the past, the job running time is usually given by users and often far exceeded the actual running time of the job, which leads to inaccurate backfilling and a waste of computing resources. In particular, when the predicted job running time is lower than the actual time, the damage caused to the utilization of the system’s computing resources becomes more serious. Therefore, the prediction accuracy of the job running time is crucial to the utilization of system resources. The use of machine learning methods can make more accurate predictions of the job running time. Aiming at the parallel application of aerodynamics, we propose a job running time prediction framework SU combining supervised and unsupervised learning and verify it on the real historical data of the high-performance computing systems of China Aerodynamics Research and Development Center (CARDC). The experimental results show that SU has a high prediction accuracy (80.46%) and a low underestimation rate (24.85%).

show abstract

“…In [10], Fan Y et al mainly started from the point of view of data processing, hoping to optimize and filter data to improve the prediction accuracy of job running time. A lot of work before this has been devoted to improving the prediction accuracy by reducing the overestimation of running time but did not solve the problem of underestimation.…”

Section: Related Workmentioning

confidence: 99%

Predicting running time of aerodynamic jobs in HPC system by combining supervised and unsupervised learning method

Wang

Dai

et al. 2021

Preprint

View full text Add to dashboard Cite

Improving resource utilization is an important goal of high-performance computing systems of supercomputing centers. In order to meet this goal, the job scheduler of high-performance computing systems often use backfilling scheduling to fill short-time jobs into the gaps of jobs at the front of the queue. Backfilling scheduling needs to obtain the running time of the job. In the past, the job running times are usually given by users and often far exceeded the actual running time of the job, which leads to inaccurate backfilling and a waste of computing resources. In particular, when the predicted job running time is lower than the actual time, the damage caused to the utilization of the system’s computing resources becomes more serious. Therefore, the prediction accuracy of the job running time is crucial to the utilization of system resources. The use of machine learning methods can make more accurate predictions of the job running time. Aiming at the parallel application of aerodynamics, we propose a job running time prediction framework SU combining supervised and unsupervised learning, and verifies it on the real historical data of the high-performance computing systems of China Aerodynamics Research and Development Center(CARDC). The experimental results show that SU has a high prediction accuracy(80.46%) and a low underestimation rate(24.85%).

show abstract

Trade-Off Between Prediction Accuracy and Underestimation Rate in Job Runtime Estimates

Cited by 46 publications

References 25 publications

Machine Learning Predictions for Underestimation of Job Runtime on HPC System

Machine Learning Predictions for Underestimation of Job Runtime on HPC System

Predicting running time of aerodynamic jobs in HPC system by combining supervised and unsupervised learning method

Predicting running time of aerodynamic jobs in HPC system by combining supervised and unsupervised learning method

Contact Info

Product

Resources

About