Analysis of Job Failure and Prediction Model for Cloud Computing Using Machine Learning

Abstract

Modern applications, such as smart cities, home automation, and eHealth, demand a new approach to improve cloud application dependability and availability. Due to the enormous scope and diversity of the cloud environment, most cloud services, including hardware and software, have encountered failures. In this study, we first analyze and characterize the behaviour of failed and completed jobs using publicly accessible traces. We have designed and developed a failure prediction model to determine failed jobs before they occur. The proposed model aims to enhance resource consumption and cloud application efficiency. Based on three publicly available traces: the Google cluster, Mustang, and Trinity, we evaluate the proposed model. In addition, the traces were also subjected to various machine learning models to find the most accurate one. Our results indicate a significant correlation between unsuccessful tasks and requested resources. The evaluation results also revealed that our model has high precision, recall, and F1-score. Several solutions, such as predicting job failure, developing scheduling algorithms, changing priority policies, or limiting re-submission of tasks, can improve the reliability and availability of cloud services.

To mitigate primary user emulation attack trajectory using cognitive single carrier frequency division multiple access approaches: Towards next generation green IoT

“…A growing number of apps and services that store and share data must be effectively protected from threats such as hacking, tampering, and unauthorized access 103 . However, PUEA still stands as the foremost issue in CRNs.…”

“…A growing number of apps and services that store and share data must be effectively protected from threats such as hacking, tampering, and unauthorized access. 103 However, PUEA still stands as the foremost issue in CRNs. ED, commonly known as radiometry or periodogram, is one of the most popular and easiest methods of spectrum sensing.…”

Causality between violations and failure factors in cloud service and its analysis methods: a survey

“…On the other hand, in [10], they analyze the relationship between task failure and resources, scheduling type, and other factors in cloud service clusters such as Mustang, and point out that task failure and resource requests are positively correlated, as well as that high-priority tasks are more likely to fail. The work [11] introduces a statistical ML framework, Hound, that interprets what system factors contribute to significant latency from the job in data centers.…”

Deep Reinforcement Learning for Workload Prediction in Federated Cloud Environments

“…While existing literature has explored resource management in cloud environments, with a focus on Virtual Machine (VM) placement [ 8 , 9 ], the research on energy efficiency and SLA preservation in the context of FCC environments has been relatively limited. With the recent boom in Machine Learning (ML), particularly Deep Learning (DL), the adoption of DL techniques has become commonplace for optimizing complex multi-dimensional problems in this domain [ 10 , 11 , 12 , 13 ]. In this paper, we propose a detailed analysis of our novel solution, Federated Cloud Workload Prediction with Deep Q-Learning (FEDQWP) for workload prediction in FCC environments using Deep Q Learning (DQL).…”