A Combined Analytical Modeling Machine Learning Approach for Performance Prediction of MapReduce Jobs in Cloud Environment

Ataie, Ehsan; Gianniti, Eugenio; Ardagna, Danilo; Movaghar, Ali

doi:10.1109/synasc.2016.072

Cited by 20 publications

(24 citation statements)

References 27 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…What is worthwhile to note is that the previous formula represents a general relation describing either closed form results, as those presented in [15], based on ML [28], or the average execution times achieved via simulation: in this paper both the latter approaches are adopted.…”

Section: Problem Statementmentioning

confidence: 99%

“…Specifically, it is the result of a ML process to get a first order approximation of the execution time of Hadoop and Spark jobs in Cloud clusters. More in details, building upon [30], which compares linear regression, Gaussian support vector regression (SVR), polynomial SVR with degree ranging between 2 and 6, and linear SVR, this paper adopts a model learned with linear SVR, following [28]. This is due to the fact that SVR with other kinds of kernel fares worse than with the linear one, whilst plain linear regression requires specific data cleaning to avoid perfect multicollinearity in the design matrix, thus making it harder to apply in the greatest generality.…”

Section: Identifying An Initial Solutionmentioning

confidence: 99%

“…In order to avoid one-VM steps, which might lead to a very slow convergence for the optimization procedure, particularly when dealing with large clusters, the optimization heuristic exploits the fact that the execution time of DIAs (as approximated by (P2f)) is inversely proportional to the allocated resources (see also [15], [22], [28]). Hence, at every iteration the application execution time is estimated as:…”

Section: The Optimization Algorithmmentioning

confidence: 99%

See 2 more Smart Citations

Optimizing Quality-Aware Big Data Applications in the Cloud

Gianniti

Ciavotta

Ardagna

2021

IEEE Trans. Cloud Comput.

Self Cite

View full text Add to dashboard Cite

The last years witnessed a steep rise in data generation worldwide and, consequently, the widespread adoption of software solutions able to support data-intensive applications. Competitiveness and innovation have strongly benefited from these new platforms and methodologies, and there is a great deal of interest around the new possibilities that Big Data analytics promise to make reality. Many companies currently engage in data-intensive processes as part of their core businesses; however, fully embracing the data-driven paradigm is still cumbersome, and establishing a production-ready, fine-tuned deployment is time-consuming, expensive, and resource-intensive. This situation calls for innovative models and techniques to streamline the process of deployment configuration for Big Data applications. In particular, the focus in this paper is on the rightsizing of Cloud deployed clusters, which represent a cost-effective alternative to installation on premises. This paper proposes a novel tool, integrated in a wider DevOps-inspired approach, implementing a parallel and distributed simulation-optimization technique that efficiently and effectively explores the space of alternative Cloud configurations, seeking the minimum cost deployment that satisfies quality of service constraints. The soundness of the proposed solution has been thoroughly validated in a vast experimental campaign encompassing different applications and Big Data platforms.

show abstract

Section: Problem Statementmentioning

confidence: 99%

Section: Identifying An Initial Solutionmentioning

confidence: 99%

Section: The Optimization Algorithmmentioning

confidence: 99%

See 1 more Smart Citation

Optimizing Quality-Aware Big Data Applications in the Cloud

Gianniti

Ciavotta

Ardagna

2021

IEEE Trans. Cloud Comput.

Self Cite

View full text Add to dashboard Cite

show abstract

“…The other values set for the input parameters of the proposed model are presented in Table 4. Most of the values used herein as input parameters of the proposed SAN model are in the range of the values considered in other related work [10,11,15,22,29,36,37,39,42]. In order to compare different situations, the proposed model was evaluated in three modes: (1) pure power saving, (2) pure rejuvenation, and (3) with simultaneous application of rejuvenation and power saving, called power-aware rejuvenation.…”

Section: Performance Evaluationmentioning

confidence: 99%

Modeling and Evaluation of Power-Aware Software Rejuvenation in Cloud Systems

2018

Self Cite

View full text Add to dashboard Cite

Long and continuous running of software can cause software aging-induced errors and failures. Cloud data centers suffer from these kinds of failures when Virtual Machine Monitors (VMMs), which control the execution of Virtual Machines (VMs), age. Software rejuvenation is a proactive fault management technique that can prevent the occurrence of future failures by terminating VMMs, cleaning up their internal states, and restarting them. However, the appropriate time and type of VMM rejuvenation can affect performance, availability, and power consumption of a system. In this paper, an analytical model is proposed based on Stochastic Activity Networks for performance evaluation of Infrastructure-as-a-Service cloud systems. Using the proposed model, a two-threshold power-aware software rejuvenation scheme is presented. Many details of real cloud systems, such as VM multiplexing, migration of VMs between VMMs, VM heterogeneity, failure of VMMs, failure of VM migration, and different probabilities for arrival of different VM request types are investigated using the proposed model. The performance of the proposed rejuvenation scheme is compared with two baselines based on diverse performance, availability, and power consumption measures defined on the system.

show abstract

“…The latter necessitates research on teletraffic loss or queueing models, either at call-level or at packet-level [2][3][4][5][6][7]. Such models not only assist in network optimization and dimensioning procedures but they may also be used in combination with machine learning techniques [8,9] or as an input to computational intelligent techniques, such as the fuzzy analytical hierarchy process techniques [10,11]. In this paper, we concentrate on call-level teletraffic loss models.…”

Section: Introductionmentioning

confidence: 99%

An Analytical Framework in OFDM Wireless Networks Servicing Random or Quasi-Random Traffic

et al. 2019

View full text Add to dashboard Cite

We consider the downlink of an orthogonal frequency division multiplexing (OFDM)-based cell that services calls from many service-classes. The call arrival process is random (Poisson) or quasi-random, i.e., calls are generated by an infinite or a finite number of sources, respectively. In order to determine congestion probabilities and resource utilization, we model the cell as a multirate loss model. Regarding the call admission, we consider the restricted accessibility, the bandwidth reservation (BR), and the complete sharing (CS) policies. In a system of restricted accessibility, a new call may be blocked even if resources do exist. In a BR system, subcarriers can be reserved in favor of calls of high subcarrier requirements. Finally, in a CS system, a new call is blocked due to resource unavailability. In all three policies, we show that there exist recursive formulas for the determination of the various performance measures. Based on simulation, the accuracy of the proposed formulas is found to be quite satisfactory.

show abstract

A Combined Analytical Modeling Machine Learning Approach for Performance Prediction of MapReduce Jobs in Cloud Environment

Cited by 20 publications

References 27 publications

Optimizing Quality-Aware Big Data Applications in the Cloud

Optimizing Quality-Aware Big Data Applications in the Cloud

Modeling and Evaluation of Power-Aware Software Rejuvenation in Cloud Systems

An Analytical Framework in OFDM Wireless Networks Servicing Random or Quasi-Random Traffic

Contact Info

Product

Resources

About