Applying Supervised Machine Learning to Predict Virtual Machine Runtime for a Non-hyperscale Cloud Provider

Perennou, Loïc; Chiky, Raja

doi:10.1007/978-3-030-28374-2_58

Cited by 1 publication

(2 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

Section: Introductionmentioning

confidence: 99%

“…Predicting resource consumption and execution time of computational tasks is crucial for such diverse applications as job management in Big Data analytics 1 or in HPC systems, 2 scientific workflow management, 3 optimization of resource allocation for virtual machines in infrastructure as a service clouds, 4 optimization of energy consumption on mobile devices, 5 or cost-effective workload scheduling. 6 In this paper, we present an experimental evaluation of various machine learning methods for predicting the execution time of computational jobs in the context of two motivating scenarios: scientific workflow management and data processing in the ALICE experiment in CERN.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Improving prediction of computational job execution times with machine learning

Balis,

Lelek,

Bodera

et al. 2023

Concurrency and Computation

View full text Add to dashboard Cite

SummaryPredicting resource consumption and run time of computational workloads is crucial for efficient resource allocation, or cost and energy optimization. In this paper, we evaluate various machine learning techniques to predict the execution time of computational jobs. For experiments we use datasets from two application areas: scientific workflow management and data processing in the ALICE experiment at CERN. We apply a two‐stage prediction method and evaluate its performance. Other evaluated aspects include: (1) comparing performance of global (per‐workflow) versus specialized (per‐job) models; (2) impact of prediction granularity in the first stage of the two‐stage method; (3) using various feature sets, feature selection, and feature importance analysis; (4) applying symbolic regression in addition to classical regressors. Our results provide new valuable insights on using machine learning techniques to predict the runtime behavior of computational jobs.

show abstract