On the dynamic resource availability in grids

Iosup, Alexandru; Jan, Mathieu; Sonmez, Ozan; Epema, Dick

doi:10.1109/grid.2007.4354112

Cited by 84 publications

(69 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…It turned out that the fraction of bursty system failures is high in distributed systems; for example, in the VAXcluster 58% of all errors and occurred in bursts and involved multiple machines [19], and in both the VAXcluster and in Grid'5000 about 30% of all failures involve multiple machines [19,11].…”

Section: Introductionmentioning

confidence: 99%

A Model for Space-Correlated Failures in Large-Scale Distributed Systems

Gallet¹,

Yigitbasi²,

Javadi³

et al. 2010

Lecture Notes in Computer Science

Self Cite

View full text Add to dashboard Cite

Distributed systems such as grids, peer-to-peer systems, and even Internet DNS servers have grown significantly in size and complexity in the last decade. This rapid growth has allowed distributed systems to serve a large and increasing number of users, but has also made resource and system failures inevitable. Moreover, perhaps as a result of system complexity, in distributed systems a single failure can trigger within a short time span several more failures, forming a group of time-correlated failures. To eliminate or alleviate the significant effects of failures on performance and functionality, the techniques for dealing with failures require good failure models. However, not many such models are available, and the available models are valid for few or even a single distributed system. In contrast, in this work we propose a model that considers groups of time-correlated failures and is valid for many types of distributed systems. Our model includes three components, the group size, the group inter-arrival time, and the resource downtime caused by the group. To validate this model, we use failure traces corresponding to fifteen distributed systems. We find that space-correlated failures are dominant in terms of resource downtime in seven of the fifteen studied systems. For each of these seven systems, we provide a set of model parameters that can be used in research studies or for tuning distributed systems. Last, as a result of our work six of the studied traces have been made available through the Failure Trace Archive

show abstract

Section: Introductionmentioning

confidence: 99%

A Model for Space-Correlated Failures in Large-Scale Distributed Systems

Gallet¹,

Yigitbasi²,

Javadi³

et al. 2010

Lecture Notes in Computer Science

Self Cite

View full text Add to dashboard Cite

show abstract

“…Nurmi et al [18] and Schroeder et al [22] report that the shape parameter is less than 1, which means that the hazard rates (the frequency a system or component fails) decrease with time. Whereas, Iosup et al [10] report that the shape parameter is greater than 1, which indicates an increasing hazard rate over time. Hence, we wanted to explore both regions for the shape parameter in our study and created two sets of reliability configurations -one set with shape parameter ranging between 0.5 and 0.9 according to [22] and the other set with shape parameter ranging between 10 and 13 according to [10].…”

Section: A Resource Reliability Modelmentioning

confidence: 98%

“…Whereas, Iosup et al [10] report that the shape parameter is greater than 1, which indicates an increasing hazard rate over time. Hence, we wanted to explore both regions for the shape parameter in our study and created two sets of reliability configurations -one set with shape parameter ranging between 0.5 and 0.9 according to [22] and the other set with shape parameter ranging between 10 and 13 according to [10]. Each set comprises of three reliability characteristics based on the quality of the resource (from a reliability standpoint) -stable, normal and shaky.…”

Section: A Resource Reliability Modelmentioning

confidence: 98%

“…Recent studies [20], [10], [22], [18] show that the mean time between failures (MTBF) on modern high performance clusters is best modeled by a Weibull distribution [25]. However, the shape and scale parameters are different for each study.…”

Section: A Resource Reliability Modelmentioning

confidence: 99%

“…At the same time, the recent growth in size and complexity of the grid infrastructure makes it susceptible to failures at all system levels -power supply, computing hardware, network, operating system, grid middleware etc. For example, the study in [10] shows that the mean time between failures (MTBF) on Grid5000 [6] is only around 12 minutes. Hence, not only is managing and scheduling workflow applications a hard problem and studied in detail [16], [27], [26], additional challenges in providing reliability to workflow executions arise because of the unreliable nature of the underlying hardware and software.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Combined Fault Tolerance and Scheduling Techniques for Workflow Applications on Computational Grids

Yang

Mandal

Koelbel

et al. 2009

2009 9th IEEE/ACM International Symposium on Cluster Computing and the Grid

View full text Add to dashboard Cite

More and more complex scientific workflows are now executed on computational grids. In addition to the challenges of managing and scheduling these workflows, additional reliability challenges arise because of the unreliable nature of large-scale grid infrastructure. Fault tolerance mechanisms like over-provisioning and checkpoint-recovery are used in current grid application management systems to address these reliability challenges. In this work, we propose new approaches that combine these fault tolerance techniques with existing workflow scheduling algorithms. We present a study on the effectiveness of the combined approaches by analyzing their impact on the reliability of workflow execution, workflow performance and resource usage under different reliability models, failure prediction accuracies and workflow application types.

show abstract

Multi‐objective optimization algorithm of adaptive entropy wireless network based on embedded DSP

Sun

Gui²

2018

Concurrency and Computation

View full text Add to dashboard Cite

Summary The dynamic changes and uncertainties of wireless networks bring a lot of difficulties to the target management and task scheduling. In order to accurately grasp the target dynamic optimization and available capacity information, a new algorithm based on adaptive entropy and multi‐objective optimization is proposed. In adaptive entropy, the objective function and constraint condition of the maximum entropy and entropy increasing principle are used to select the objective which satisfies the maximization of the system, and the optimal scheduling is carried out, and then the multi‐objective optimization is carried out on the selected target, to migrate, release the target of heavy optimization and long‐term unavailability, which can reduce the energy consumption, realize the optimization balance, and improve the system utilization rate. The simulation experiment is designed to verify the performance of the proposed algorithm. The experimental results show that the adaptive entropy algorithm is very effective for user QoS and system maximization, which enhances the system utilization rate. This optimization algorithm has realized the user QoS guarantee, reduces the energy consumption, the optimized equilibrium, the enhanced system utilization, and so on many optimization goals.

show abstract

On the dynamic resource availability in grids

Cited by 84 publications

References 21 publications

A Model for Space-Correlated Failures in Large-Scale Distributed Systems

A Model for Space-Correlated Failures in Large-Scale Distributed Systems

Combined Fault Tolerance and Scheduling Techniques for Workflow Applications on Computational Grids

Multi‐objective optimization algorithm of adaptive entropy wireless network based on embedded DSP

Contact Info

Product

Resources

About