A Two-Stage Framework for the Multi-User Multi-Data Center Job Scheduling and Resource Allocation

Lin, Jianpeng; Cui, Delong; Peng, Zhiping; Li, Qirui; He, Jiping

doi:10.1109/access.2020.3033557

Cited by 30 publications

(21 citation statements)

References 42 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The proposed methodologies' efficiency is evaluated in terms of computational time and the results obtained are presented in Table 2. The techniques taken for comparison are Q‐learning, 34 CSLB, 27 RoFFR, 35 GWO‐RIL, 37 ACO, 38 HDDL‐DQN, 39 and DRL 40 . The computational time of the proposed DRQL is 13.2 seconds which is lower when compared to the existing techniques.…”

Section: Resultsmentioning

confidence: 99%

“…The optimization approach is mainly applied to the MapReduce application to minimize the additional overhead created while big data processing. Lin et al 39 presented a two‐stage framework for multi‐user and data center job scheduling and resource allocation. Here, the heterogeneous distributed deep learning (HDDL) model is used for job scheduling and the deep Q‐network (DQN) model is used for resource allocation.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Optimized intellectual resource scheduling using deep reinforcement Q‐learning in cloud computing

Uma

Vivekanandan²,

Shankar

2022

Trans Emerging Tel Tech

View full text Add to dashboard Cite

Cloud computing has recently attracted both academics and industrialists in the field of research. Virtualization allows cloud service providers (CSPs) with their own data centers to supply infrastructures, resources, and services to users by converting real servers into virtual machines (VMs). Profit‐driven CSPs charge users for VM leasing and service access while reducing energy consumption to increase profits. But CSPs even face challenges like minimizing the energy cost for the data center. Several different algorithms were introduced for minimizing the energy cost by using task scheduling (TS) and/or resource provisioning. However, scalability issues were encountered, or TS with task dependencies were not considered, which is a critical factor in assuring exact parallel execution of tasks in parallel. This article introduces a novel artificial algorithm, called deep reinforcement Q‐learning for resource scheduling which integrates the features of the Q‐learning and reinforcement learning approaches. The objective of this new approach is to provide a solution to the problem of handling energy consumption in a cloud computing environment. Based on advancements in WorkflowSim, experiments are carried out comparatively by considering the variance of make‐span, time, cost analysis, deadline overflow, and load balance in resource scheduling. The proposed method tends to be effective in terms of cost, energy consumption, resource utilization, and response time. The resource reuse capability of the proposed methodology is 63% higher when compared to the modified particle swarm optimization and modified cat swarm optimization technique. The task approval rate of the proposed methodology is 54% higher than the crow search‐based load balancing algorithm and 50% higher than the task duplication‐based scheduling algorithm.

show abstract

Section: Resultsmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Optimized intellectual resource scheduling using deep reinforcement Q‐learning in cloud computing

Uma

Vivekanandan²,

Shankar

2022

Trans Emerging Tel Tech

View full text Add to dashboard Cite

show abstract

“…Compared to the Non-Dominated Sorting Genetic Algorithm, Extreme Non-Dominated Sorting Genetic Algorithm-III the performance of Multilevel-Dependent Node Clustering is better in terms of cost and makespan. Lin et al (2020) used deep learning techniques for effective job scheduling to multiple cloud data centers and a deep Q network for resource allocation to reduce energy consumption and delay, thereby increasing the performance of a multicloud environment. This combined approach eliminates the limitations and complexities in the traditional scheduling models.…”

Section: Related Workmentioning

confidence: 99%

Multi-swarm optimization model for multi-cloud scheduling for enhanced quality of services

Mohanraj

Santhosh

2021

Soft Comput

View full text Add to dashboard Cite

Cloud services gain more attention due to its accessibility, performance, and cost factors. Cloud offers a wide range of services and completes the task without any delay due to its scheduling policies. Task scheduling is an important factor in cloud computing applications. The performance of applications increases due to an effective scheduling strategy. The cloud resources are allocated to the tasks through task scheduling. Factors like customer satisfaction, resource utilization, better performance make task scheduling crucial for service providers. Depending on the scheduling schemes support in clouds, scheduling is categorized into single cloud or multi-cloud scheduling. Multi-cloud environment provides diverse resources and significantly reduces the cost and commercial limitations. However, reducing the cost functions and makespan are the major factors considered to avoid customer dissatisfaction. But it is essential to concentrate on other factors, such as throughput, delay, Makespan, waiting time, response time, utilization, and efficiency to improve the quality of services. This research work presents a Multi-Swarm Optimization model for Multi-Cloud Scheduling for Enhanced Quality of Services for a multi-cloud environment. Experimental results demonstrate that the proposed approach performs better in all aspects compared to existing techniques, such as Adaptive energy-efficient scheduling, single objective particle swarm optimization scheduling, and improves the quality of services.

show abstract

“…Lin et al [20] proposed a two-stage scheduling and resource allocation framework that leverages machine learning models to schedule jobs to multiple data centers compared to our work which focuses on a single system. Marahatta et al [24] described an approach that uses machine learning models to classify incoming tasks as either "failure-prone" or "non-failure-prone" tasks.…”

Section: Related Workmentioning

confidence: 99%

Dynamic Scheduling for Heterogeneous Multicores

et al. 2021

View full text Add to dashboard Cite

Heterogeneous multicore systems can help adherence to design goals by providing a diverse set of hardware components to meet application requirements. Each core may also have tunable hardware that can reconfigured for different applications. However, scheduling becomes difficult in the presence of tunable hardware due to the additional constraint that an application must be scheduled to a core that offers the best configuration, based on the application's requirements, to maximize the benefit of the tunable hardware. The scheduling decision can be made by exploring and evaluating each hardware configuration in the design space exhaustively or through the use of a heuristic to evaluate a subset of the design space. However, exhaustive and heuristic approaches do not scale well with the number of available hardware configurations. To improve upon exhaustive and heuristic approaches, in this paper, we present a dynamic scheduling methodology that uses machine learning to schedule applications to the application's respective best core for reduced energy consumption for a system with configurable caches. We use an artificial neural network (ANN) to train our predictive model, using hardware counters, to predict the best core and a tuning heuristic to determine best configuration on non-best cores. If the best core is busy, our scheduler considers alternative idle cores, or the application is stalled depending on which decision is energy advantageous. Our experiments show that our proposed system can achieve a total energy savings of approximately 33% and 2.5% compared to a fixed-cache, base system when using a configurable instruction and data cache, respectively, while remaining competitive and exhibiting better scaling properties compared to systems adopting a heuristic approach.

show abstract

A Two-Stage Framework for the Multi-User Multi-Data Center Job Scheduling and Resource Allocation

Cited by 30 publications

References 42 publications

Optimized intellectual resource scheduling using deep reinforcement Q‐learning in cloud computing

Optimized intellectual resource scheduling using deep reinforcement Q‐learning in cloud computing

Multi-swarm optimization model for multi-cloud scheduling for enhanced quality of services

Dynamic Scheduling for Heterogeneous Multicores

Contact Info

Product

Resources

About