Predicting queue times on space-sharing parallel computers

Downey, Allen B.

doi:10.1109/ipps.1997.580894

Cited by 112 publications

(89 citation statements)

References 8 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Nevertheless, it was observed a tendency for more higher prediction errors as the σ 2 value increases. The Table 3 shows the best IBL model prediction errors obtained compared to five previous approaches [12] [3] [4](the mean absolute error is used in order to compare the results with previous work). …”

Section: Global Knowledge Acquisitionmentioning

confidence: 99%

Using Runtime Measurements and Historical Traces for Acquiring Knowledge in Parallel Applications

Senger

Santana

2004

Computational Science - ICCS 2004

View full text Add to dashboard Cite

Abstract.A new approach for acquiring knowledge of parallel applications regarding resource usage and for searching similarity on workload traces is presented. The main goal is to improve decision making in distributed system software scheduling, towards a better usage of system resources. Resource usage patterns are defined through runtime measurements and a self-organizing neural network architecture, yielding an useful model for classifying parallel applications. By means of an instance-based algorithm, it is produced another model which searches for similarity in workload traces aiming at making predictions about some attribute of a new submitted parallel application, such as run time or memory usage. These models allow effortless knowledge updating at the occurrence of new information. The paper describes these models as well as the results obtained applying these models to acquiring knowledge in both synthetic and real applications traces.

show abstract

Section: Global Knowledge Acquisitionmentioning

confidence: 99%

Using Runtime Measurements and Historical Traces for Acquiring Knowledge in Parallel Applications

Senger

Santana

2004

Computational Science - ICCS 2004

View full text Add to dashboard Cite

show abstract

“…[221]), and it is also possible to create new constructions based on the concepts described here. For example, Downey has defined the log-uniform distribution to describe the runtimes of jobs on parallel supercomputers [186,187]. This is inspired by the lognormal distribution: first perform a logarithmic transformation, and then see what the resulting distribution looks like.…”

Section: Do It Yourselfmentioning

confidence: 99%

Workload Modeling for Computer Systems Performance Evaluation

Feitelson

2015

196

131

View full text Add to dashboard Cite

Reliable performance evaluations require the use of representative workloads. This is no easy task since modern computer systems and their workloads are complex, with many interrelated attributes and complicated structures. Experts often use sophisticated mathematics to analyze and describe workload models, making these models difficult for practitioners to grasp. This book aims to close this gap by emphasizing the intuition and the reasoning behind the definitions and derivations related to the workload models. It provides numerous examples from real production systems, with hundreds of graphs. Using this book, readers will be able to analyze collected workload data and clean it if necessary, derive statistical models that include skewed marginal distributions and correlations, and consider the need for generative models and feedback from the system. The descriptive statistics techniques covered are also useful for other domains.

show abstract

“…The statistical method by Downey [13] used the observation that the cumulative distributions of the execution times of the jobs in the workload can be modeled by using a logarithmic function. After the distribution functions are calculated, two different methods are used to predict when a certain number of nodes will become free and thus when the job waiting at the head of the queue can start.…”

Section: Related Workmentioning

confidence: 99%

Identifying Quick Starters: Towards an Integrated Framework for Efficient Predictions of Queue Waiting Times of Batch Parallel Jobs

Kumar

Vadhiyar

2013

Job Scheduling Strategies for Parallel Processing

View full text Add to dashboard Cite

Abstract. Production parallel systems are space-shared and hence employ batch queues in which the jobs submitted to the systems are made to wait before execution. Thus, jobs submitted to parallel batch systems incur queue waiting times in addition to the execution times. Prediction of these queue waiting times is important to provide overall estimates to the users and can also help metaschedulers make scheduling decisions. Analyses of the job traces of supercomputers reveal that about 56 to 99% of the jobs incur queue waiting times of less than an hour. Hence, identifying these quick starters or jobs with short queue waiting times is essential for overall improvement on queue waiting time predictions. Existing strategies provide high overestimates of upper bounds of queue waiting times rendering the bounds less useful for jobs with short queue waiting times. In this work, we have developed an integrated framework that uses the job characteristics, and states of the queue and processor occupancy to identify and predict quick starters, and use the existing strategies to predict jobs with long queue waiting times. Our experiments with different production supercomputer job traces show that our prediction strategies can lead to correct identification of up to 20 times more quick starters and provide tighter bounds for these jobs, and thus result in up to 64% higher overall prediction accuracy than existing methods.

show abstract

Predicting queue times on space-sharing parallel computers

Cited by 112 publications

References 8 publications

Using Runtime Measurements and Historical Traces for Acquiring Knowledge in Parallel Applications

Using Runtime Measurements and Historical Traces for Acquiring Knowledge in Parallel Applications

Workload Modeling for Computer Systems Performance Evaluation

Identifying Quick Starters: Towards an Integrated Framework for Efficient Predictions of Queue Waiting Times of Batch Parallel Jobs

Contact Info

Product

Resources

About