Job failure prediction in grid environment based on workload characteristics

Fadishei, Hamid; Saadatfar, Hamid; Deldari, Hossein

doi:10.1109/csicc.2009.5349381

Cited by 28 publications

(13 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Fadishei el al. [12] analyze the correlations between job failures and job attributes such as CPU utilization and memory usage based on Grid Workload Trace. They then utilize a simplified Bayesian network to predict job failures in grid computing system.…”

Section: B Online Failure Predictionmentioning

confidence: 99%

Time Series Based Killer Task Online Recognition Service: A Google Cluster Case Study

Tang

Liu

Jia

et al. 2016

2016 IEEE Symposium on Service-Oriented System Engineering (SOSE)

View full text Add to dashboard Cite

To better understand task failures in cloud computing systems, we analyze failure frequency of tasks based on Google cluster dataset, and find what we call as killer tasks that suffer from long-term failures and repeated rescheduling. Killer task can be a big concern of cloud systems as it causes unnecessary resource wasting and significant increase of scheduling workloads. Hence there is a need to provide a service for cloud system operators to recognize killer tasks in time. In this paper, we propose an online killer task recognition service based on the resource usage time series which can recognize killer tasks at the very early stage of their occurrence so that they can be handled appropriately instead of being rescheduled. The experiment results show that the proposed service performs a 93.6% accuracy in recognizing killer tasks with an 87% timing advance and 86.6% resource saving for the cloud system averagely.

show abstract

Section: B Online Failure Predictionmentioning

confidence: 99%

Time Series Based Killer Task Online Recognition Service: A Google Cluster Case Study

Tang

Liu

Jia

et al. 2016

2016 IEEE Symposium on Service-Oriented System Engineering (SOSE)

View full text Add to dashboard Cite

show abstract

“…Clouds also experience dynamic workload size and variety, from a result of the Clouds ability to quickly scale to a consumer's needs as well as the Cloud's multi-tenancy. It has been established that different workload properties such as computationally and I/O intensive workloads influence the type of faults that are activated [18].…”

Section: B Dependability In Cloudmentioning

confidence: 99%

Real-Time Fault-Tolerance in Federated Cloud Environments

Garraghan

Townend

2012

2012 IEEE 15th International Symposium on Object/Component/Service-Oriented Real-Time Distributed Computing Workshops

View full text Add to dashboard Cite

Universities of LeedsAbstract-Dependability is a critical concern in provisioning services in Cloud Computing environments. This is true when considering reliability, an attribute of dependability that is a critical and challenging problem in a Cloud context [2]. Faulttolerance is one means to attain reliability, and is typically implemented by using some form of diversity. Federated Cloud, which is an emerging Cloud paradigm that orchestrates multiple Clouds, is able to implement environmental diversity for Cloud applications with relative ease and minimal additional cost to the consumer due to its inherent design. Real-Time Applications (RTAs) can benefit from deploying fault-tolerant schemes to fulfill deadlines in the presence of faults as they enable the provisioning of correct service in the event of a component in the application failing. However, this diversity can potentially become an issue when designing dynamically scalable faulttolerant RTAs in a federated Cloud environment while also fulfilling QoS demands. In particular, building fault-tolerant RTAs by using the diversity of the Virtual Machine (VM) configurations and of the underlying Cloud infrastructure can have a negative impact on the ability to fulfill deadlines whilst still allowing the application to dynamically provision VMs with minimal human interaction. This paper identifies a number of characteristics that affect the ability for a RTA to fulfill specified deadlines in a federated Cloud environment as a result of deploying environment diverse fault-tolerant schemes. Furthermore we have designed and performed initial experiments using a real world Cloud federation to justify the feasibility of this problem. Results demonstrate that deploying RTAs in a federated Cloud environment can potentially increase the rate of deadline violations.

show abstract

“…Therefore, although they have an acceptable foresight for resource failures and to some extent can prevent from occurrence of job failures they don't notice to vital correlation of features like jobs' requirements and characteristics (submission hour, requested memory, needed processing element), user's specifications (user identification), and system situations (system load, nodes' availability rate), which can also cover more other failure causes and help to have a better job futurity predictor [2].…”

Section: Rela Ted Workmentioning

confidence: 99%

“…Building the prediction model has the same process as [2]. In the following, the training data set and the learning method are explained.…”

Section: Job Futurity Predictormentioning

confidence: 99%

“…Although these systems can address the need of tasks such as economic forecasting and drug discovery for petatlops of computing power and petabytes of storage capacity, there is a simple fact that challenges the researchers who try to harness the computing power of them: large-scale systems are fault prone environments and have high job failure rate [2]. In fact, in such systems, a significant percent (near to 30) of submitted jobs is affected by failures and therefore they can't finish their execution [3 ], [4].…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

JSM: Job submission manager for large-scale distributed systems based on game theory

Saadatfar

Salami

Deldari

et al. 2010

2010 3rd International Conference on Advanced Computer Theory and Engineering(ICACTE)

Self Cite

View full text Add to dashboard Cite

A bstract-Nowadays by improving the richness of prediction methods and accessing to the more information about systems behavior, the role of proactive strategies in developing more reliable and efficient systems becomes more crucial. However, the goals of prediction and the way that the results can be employed to upgrade the system are still topics which draw recent researchers' attention. In this work, we attempt to decrease Jobs wait time and failure rate by using the results of a job futurity predictor. For achieving this goal, a system component called JSM is proposed. JSM consults the predictor and employs a game theory based model in order to probably rejecting the jobs which are likely to fail. Furthermore, for avoiding from rejecting safety jobs mistakenly, JSM intelligently adopts its decisions with the systems situations. Experimental results state a significant reduction in jobs wait time and failure rate in comparison with other related work.Keywords-job submission manager; cluster computing; job futurity prediction; game theory 1.978-1-4244-6542-2/$26.00

show abstract

Job failure prediction in grid environment based on workload characteristics

Cited by 28 publications

References 15 publications

Time Series Based Killer Task Online Recognition Service: A Google Cluster Case Study

Time Series Based Killer Task Online Recognition Service: A Google Cluster Case Study

Real-Time Fault-Tolerance in Federated Cloud Environments

JSM: Job submission manager for large-scale distributed systems based on game theory

Contact Info

Product

Resources

About