Competitive online adaptive scheduling for sets of parallel jobs with fairness and efficiency

Sun, Hongyang; Hsu, Wen-Jing; Cao, Yangjie

doi:10.1016/j.jpdc.2013.12.003

Cited by 7 publications

(4 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Others researchers also give more work to parallel tasks in different aspects, such as fairness [11,12], efficiency [11], energy consuming [13,14]. They are not the target of our paper and we do not give more information about those studies.…”

Section: Related Workmentioning

confidence: 97%

An adaptive algorithm for scheduling parallel jobs in meteorological Cloud

Hao

Wang

Zheng

2016

Knowledge-Based Systems

View full text Add to dashboard Cite

From traditional clusters to cloud systems, job scheduling is one of the most critical factors for achieving high performance in any distributed environment. In this paper, we propose an adaptive algorithm for scheduling modular non-linear parallel jobs in meteorological Cloud, which has a unique parallelism that can only be configured at the very beginning of the execution. Different from existing work, our algorithm takes into account four characteristics of the jobs at the same time, including the average execution time, the deadlines of jobs, the number of assigned resources, and the overall system loads. We demonstrate the effectiveness and efficiency of our scheduling algorithm through simulations using WRF (Weather Research and Forecasting model) that which is widely used in scientific computing. Our evaluation results show that the proposed algorithm has multiple advantages compared with previous methods, including more than 10% reduction in terms of execution time, a higher completion ratio in terms of meeting soft deadlines, and a much smaller standard deviation of the average weighted execution time. Moreover, we show that the proposed algorithm can tolerate inaccuracy in system load estimation.

show abstract

Section: Related Workmentioning

confidence: 97%

An adaptive algorithm for scheduling parallel jobs in meteorological Cloud

Hao

Wang

Zheng

2016

Knowledge-Based Systems

View full text Add to dashboard Cite

show abstract

“…For the makespan minimization problem, it was shown in [30] that EQUI achieves a competitive ratio of O( ln n ln ln n ) when jobs are organized in two levels, where n is the total number of jobs in the system, and that no better ratio is possible. Two closely related work to ours in a similar setting are by Robert et al [30] and Sun et al [31,32]. In [30], the authors considered a three-level hierarchy by organizing the jobs in different job sets and present an online scheduling algorithm EQUI•EQUI, which first splits evenly the available processors among the job sets, and then splits evenly the allocated processors among the jobs of each set.…”

Section: Related Workmentioning

confidence: 99%

“…The proposed algorithm were shown to achieve O(1)-competitiveness. Finally, Sun et al [32] generalized the result to an arbitrary number of hierarchical levels for the metric of set response time .…”

Section: Related Workmentioning

confidence: 99%

Scalable Hierarchical Scheduling for Malleable Parallel Jobs on Multiprocessor-based Systems

Cao¹,

Sun²,

Qian³

et al. 2014

Preprint

Self Cite

View full text Add to dashboard Cite

The proliferation of multi-core and multiprocessor-based computer systems has led to explosive development of parallel applications and hence the need for efficient schedulers. In this paper, we study hierarchical scheduling for malleable parallel jobs on multiprocessor-based systems, which appears in many distributed and multilayered computing environments. We propose a hierarchical scheduling algorithm, named AC-DS, that consists of a feedback-driven adaptive scheduler, a desire aggregation scheme and an efficient resource allocation policy. From theoretical perspective, we show that AC-DS has scalable performance regardless of the number of hierarchical levels. In particular, we prove that AC-DS achieves O(1)-competitiveness with respect to the overall completion time of the jobs, or the makespan. A detailed malleable job model is developed to experimentally evaluate the effectiveness of the proposed scheduling algorithm. The results verify the scalability of AC-DS and demonstrate that AC-DS outperforms other strategies for a wide range of parallel workloads.

show abstract

“…Fairness based on sharing of resources such as processors, memory, system clock and system bus in multi-programmed multi-user system was well studied in [9][10][11]. Some recent works with contributions on algorithmic fairness of online scheduling can be found in [17][18]. According to our knowledge, study of fairness based on user's objective has not been exhaustively studied in the literature.…”

Section: Introductionmentioning

confidence: 99%

A New Fairness Model based on User's Objective for Multi-user Multi-processor Online Scheduling

Dwibedy,

Mohanty

2020

Preprint

View full text Add to dashboard Cite

Resources of a multi-user system in multi-processor online scheduling are shared by competing users in which fairness is a major performance criterion for resource allocation. Fairness ensures equality in resource sharing among the users. According to our knowledge, fairness based on the user's objective has neither been comprehensively studied nor a formal fairness model has been well defined in the literature. This motivates us to explore and define a new model to ensure algorithmic fairness with quantitative performance measures based on optimization of the user's objective. In this paper, we propose a new model for fairness in Multi-user Multi-processor Online Scheduling Problem(MUMPOSP). We introduce and formally define quantitative fairness measures based on user's objective by optimizing makespan for individual user in our proposed fairness model. We also define the unfairness of deprived users and absolute fairness of an algorithm. We obtain lower bound results for the absolute fairness for m-identical machines with equal length jobs. We show that our proposed fairness model can serve as a framework for measuring algorithmic fairness by considering various optimality criteria such as flow time and sum of completion times.

show abstract

Competitive online adaptive scheduling for sets of parallel jobs with fairness and efficiency

Cited by 7 publications

References 24 publications

An adaptive algorithm for scheduling parallel jobs in meteorological Cloud

An adaptive algorithm for scheduling parallel jobs in meteorological Cloud

Scalable Hierarchical Scheduling for Malleable Parallel Jobs on Multiprocessor-based Systems

A New Fairness Model based on User's Objective for Multi-user Multi-processor Online Scheduling

Contact Info

Product

Resources

About