Utility-based resource management in an oversubscribed energy-constrained heterogeneous environment executing parallel applications

Machovec, Dylan; Khemka, Bhavesh; Kumbhare, Nirmal; Pasricha, Sudeep; Maciejewski, Anthony A.; Siegel, Howard Jay; Akoglu, Ali; Koenig, Gregory A.; Hariri, Salim; Tunc, Cihan; Wright, Michael; Hilton, Marcia; Rambharos, Rajendra; Blandin, Christopher; Fargo, Farah; Louri, Ahmed; Imam, Neena

doi:10.1016/j.parco.2017.11.005

Cited by 10 publications

(3 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We are currently addressing challenges of VDCs management on simpler environments, on cloud resource management heuristics, (e.g., [8,9,6,7]), big data analysis, and data mining for performance prediction. To simulate, evaluate, analyze, and compare different heuristics, we will build simulators for simpler environments and combine open-source simulators for different levels of the JITA-4DS hierarchy.…”

Section: Discussionmentioning

confidence: 99%

Graph Analytics Workflows Enactment on Just in Time Data Centres Position Paper

Akoglu

Zechinelli-Martini

Kheddouci

et al. 2022

Service-Oriented Computing – ICSOC 2021 Workshops

Self Cite

View full text Add to dashboard Cite

This paper discusses our vision about multirole-capable decisionmaking systems across a broad range of Data Science (DS) workflows working on graphs through disaggregated data centres. Our vision is that an alternative is possible, to work on a disaggregated solution for the provision of computational services under the notion of a disaggregated data centre. We define this alternative as a virtual entity that dynamically provides resources crosscutting the layers of edge, fog and data centre according to the workloads submitted by the workflows and their Service Level Objectives. This work is funded by the project GALILEAN, LIRIS intergroup collaboration https://galilean-project.github.io.

show abstract

Section: Discussionmentioning

confidence: 99%

Graph Analytics Workflows Enactment on Just in Time Data Centres Position Paper

Akoglu

Zechinelli-Martini

Kheddouci

et al. 2022

Service-Oriented Computing – ICSOC 2021 Workshops

Self Cite

View full text Add to dashboard Cite

show abstract

“…This ability is crucial for meeting the SLO of edge-based data science pipeline execution, where the nature and amount of the data change dynamically among streams of data arriving from heterogeneous and numerous sets of edge devices. In our earlier work [13], we defined the value for a task as a monotonically-decreasing function of an objective (i.e., completion time, energy), illustrated in Figure 3. The soft threshold parameter specifies the limit on an objective (T h sof t ) until which the value earned by a task is maximum (v max ).…”

Section: Value Of Service Based Scheduling and Resource Managementmentioning

confidence: 99%

Putting Data Science Pipelines on the Edge

Akoglu¹,

Vargas-Solar²

2021

Preprint

Self Cite

View full text Add to dashboard Cite

This paper proposes a composable "Just in Time Architecture" for Data Science (DS) Pipelines named JITA-4DS and associated resource management techniques for configuring disaggregated data centers (DCs). DCs under our approach are composable based on vertical integration of the application, middleware/operating system, and hardware layers customized dynamically to meet application Service Level Objectives (SLO -application-aware management). Thereby, pipelines utilize a set of flexible building blocks that can be dynamically and automatically assembled and re-assembled to meet the dynamic changes in the workload's SLOs. To assess disaggregated DC's, we study how to model and validate their performance in large-scale settings.

show abstract

“…ITA-4DS benefits from virtualization techniques in the following aspects. Virtual Machine (VM) technologies such as VMware [11] and Xen [9] provide a flexible management platform useful for both the encapsulation of application execution environments and the aggregation and accounting of resources consumed by an application [24]. Consequently, VMs have been widely used to provide a layer that is well-positioned in the hardware/software stack of computer systems.…”

Section: Jita-4ds: Just In Time Edge Based Data Science Pipelines Exe...mentioning

confidence: 99%

JITA4DS: Disaggregated Execution of Data Science Pipelines Between the Edge and the Data Centre

Vargas-Solar

Akoglu

Hassan

2021

JWE

View full text Add to dashboard Cite

This paper targets the execution of data science (DS) pipelines supported by data processing, transmission and sharing across several resources executing greedy processes. Current data science pipelines environments provide various infrastructure services with computing resources such as general-purpose processors (GPP), Graphics Processing Units (GPUs), Field Programmable Gate Arrays (FPGAs) and Tensor Processing Unit (TPU) coupled with platform and software services to design, run and maintain DS pipelines. These one-fits-all solutions impose the complete externalization of data pipeline tasks. However, some tasks can be executed in the edge, and the backend can provide just in time resources to ensure ad-hoc and elastic execution environments.This paper introduces an innovative composable “Just in Time Architecture” for configuring DCs for Data Science Pipelines (JITA-4DS) and associated resource management techniques. JITA-4DS is a cross-layer management system that is aware of both the application characteristics and the underlying infrastructures to break the barriers between applications, middleware/operating system, and hardware layers. Vertical integration of these layers is needed for building a customizable Virtual Data Center (VDC) to meet the dynamically changing data science pipelines’ requirements such as performance, availability, and energy consumption. Accordingly, the paper shows an experimental simulation devoted to run data science workloads and determine the best strategies for scheduling the allocation of resources implemented by JITA-4DS.

show abstract

Utility-based resource management in an oversubscribed energy-constrained heterogeneous environment executing parallel applications

Abstract: Wright, M. (2017). Utilitybased resource management in an oversubscribed energyconstrained heterogeneous environment executing parallel applications. Parallel Computing. (Cihan Tunc), michael.wright4@comcast.net (Michael Wright), mmskizig@verizon.net (Marcia Hilton),

Cited by 10 publications

References 23 publications

Graph Analytics Workflows Enactment on Just in Time Data Centres Position Paper

Graph Analytics Workflows Enactment on Just in Time Data Centres Position Paper

Putting Data Science Pipelines on the Edge

JITA4DS: Disaggregated Execution of Data Science Pipelines Between the Edge and the Data Centre

Contact Info

Product

Resources

About