Cloud Failure Prediction based on Traditional Machine Learning and Deep Learning

Abstract

Cloud failure is one of the critical issues since it can cost millions to the cloud service providers in addition to the loss of productivity being suffered by the industrial users using different cloud-based applications and services. As the cloud system grows larger, the process of failure prediction is still a challenge for both practitioners and academic researchers. In this study, we tackle the challenge of predicting job and task failure using the benchmarking Google Cluster Traces Dataset. We first propose a conceptual model to prepare, construct and evaluate five traditional machine learning algorithms and three variants of the latest deep learning algorithms for predicting job and task failures. We then perform a series of experiments on two datasets; (1) Dataset A for the job failure prediction models and (2) Dataset B for the task failure prediction models. In the case of job failure prediction experiments, we observe that Extreme Gradient Boosting is the best performance with 94.35% accuracy with the F-Score score of 0.9310, 91.92% sensitivity, and 96.07% specificity. For Extreme Gradient Boosting, the disk space request and CPU request are the most important features in determining the outcome of the job prediction. In the case of task failure prediction, we observe that Decision Tree and Random Forest have achieved the highest 89.75% accuracy and 0.9145 for F-Score with 98% sensitivity and 78% specificity. The priority of the task is the most important feature for determining the task prediction outcome for both, the Decision Tree and Random Forest.