D-SPACE4Cloud: A Design Tool for Big Data Applications

Ciavotta, Michele; Gianniti, Eugenio; Ardagna, Danilo

doi:10.1007/978-3-319-49583-5_48

Cited by 9 publications

(7 citation statements)

References 31 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Therefore, in order to make the D-SPACE4Cloud usable, we focused on increasing the level of parallelism as much as possible. In [4] we have shown that, in the worst case, the relative error between performance prediction models and real applications execution can reach up to 32.97%, which is perfectly in line with the expected accuracy in the performance prediction field [8] while the average relative error is 14.13% overall.…”

Section: D-space4cloudsupporting

confidence: 75%

“…D-SPACE4Cloud [4] is the tool we used to evaluate the cost impact of privacy mechanisms implementation. D-SPACE4Cloud supports the capacity planning process of shared Hadoop Cloud clusters for MapReduce or Spark applications with deadline guarantees.…”

Section: D-space4cloudmentioning

confidence: 99%

“…In particular, masking and encryption techniques are applied to a real industry case study in the tax fraud detection application domain and an extensive experimental campaign has been performed to analyze the efficiency of a Spark cloud based clusters against these techniques. Moreover, relying on our D-SPACE4Cloud tool [4] (which supports the capacity planning of Spark cloud clusters providing deadlines guarantees) also the cost impact of the privacy preserving mechanisms is evaluated.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Performance Degradation and Cost Impact Evaluation of Privacy Preserving Mechanisms in Big Data Systems

Kalwar

Gianniti

Kinouani³

et al. 2018

Communications in Computer and Information Science

Self Cite

View full text Add to dashboard Cite

Big Data is an emerging area and concerns managing datasets whose size is beyond the ability of commonly used software tools to capture, process, and perform analyses in a timely way. The Big Data software market is growing at 32% CAR, almost four times more than the whole ICT market, and the quantity of data to be analyzed is expected to double every two years. Security and privacy are becoming very urgent Big Data aspects that need to be tackled. Indeed, users share more and more personal data and user generated content through their mobile devices and computers to social networks and cloud services, losing data and content control with a serious impact on their own privacy. Privacy is one area that had a serious debate recently, and many governments require to data providers and companies to protect the users' sensitive data. To mitigate these problems, many solutions have been developed to provide data privacy, which, however, introduce some computational overhead when data is processed. The goal of this paper is to quantitatively evaluate the performance and cost impact of multiple privacy protection mechanisms. A real industry case study concerning tax fraud detection has been considered. Many experiments have been performed to analyze the performance degradation and additional cost (required to provide a given service level) for running applications in a cloud system.

show abstract

Section: D-space4cloudsupporting

confidence: 75%

Section: D-space4cloudmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Performance Degradation and Cost Impact Evaluation of Privacy Preserving Mechanisms in Big Data Systems

Kalwar

Gianniti

Kinouani³

et al. 2018

Communications in Computer and Information Science

Self Cite

View full text Add to dashboard Cite

show abstract

“…The risk tolerance knob introduced in ExoSphere allows easy and explicit characterization of portfolio risk. Transient server selection policies in earlier systems [20,36,43,45,49,57] do not have explicit support for managing revocation risk. This is because these systems have mostly targeted a single class of applications, and have server selection policies suited to that.…”

Section: Related Workmentioning

confidence: 99%

Portfolio-driven Resource Management for Transient Cloud Servers

Sharma

Irwin

Shenoy

2017

Proc. ACM Meas. Anal. Comput. Syst.

View full text Add to dashboard Cite

Cloud providers have begun to offer their surplus capacity in the form of low-cost transient servers, which can be revoked unilaterally at any time. While the low cost of transient servers makes them attractive for a wide range of applications, such as data processing and scientific computing, failures due to server revocation can severely degrade application performance. Since different transient server types offer different cost and availability tradeoffs, we present the notion of server portfolios that is based on financial portfolio modeling. Server portfolios enable construction of an "optimal" mix of severs to meet an application's sensitivity to cost and revocation risk. We implement model-driven portfolios in a system called ExoSphere, and show how diverse applications can use portfolios and application-specific policies to gracefully handle transient servers. We show that ExoSphere enables widely-used parallel applications such as Spark, MPI, and BOINC to be made transiency-aware with modest effort. Our experiments show that allowing the applications to use suitable transiency-aware policies, ExoSphere is able to achieve 80% cost savings when compared to on-demand servers and greatly reduces revocation risk compared to existing approaches. ACM Reference format:Prateek Sharma, David Irwin, and Prashant Shenoy. 2017. Portfolio-driven Resource Management for Transient Cloud Servers.

show abstract

“…Our work is one of the first contributions facing the design time problem of rightsizing data-intensive Cloud systems adopting the Capacity Scheduler. In particular, it builds upon our previous research presented in [20] and provides a thorough description of D-SPACE4Cloud 1 , a software tool designed to help system administrators and operators in the capacity planning of shared Big Data clusters hosted in the Cloud to support both batch and interactive applications with deadline guarantees. With respect to previous releases, the tool now supports, besides classical MapReduce workloads, also Spark applications, which feature generic DAGs execution models.…”

Section: Introductionmentioning

confidence: 99%

Optimizing Quality-Aware Big Data Applications in the Cloud

Gianniti

Ciavotta

Ardagna

2021

IEEE Trans. Cloud Comput.

Self Cite

View full text Add to dashboard Cite

The last years witnessed a steep rise in data generation worldwide and, consequently, the widespread adoption of software solutions able to support data-intensive applications. Competitiveness and innovation have strongly benefited from these new platforms and methodologies, and there is a great deal of interest around the new possibilities that Big Data analytics promise to make reality. Many companies currently engage in data-intensive processes as part of their core businesses; however, fully embracing the data-driven paradigm is still cumbersome, and establishing a production-ready, fine-tuned deployment is time-consuming, expensive, and resource-intensive. This situation calls for innovative models and techniques to streamline the process of deployment configuration for Big Data applications. In particular, the focus in this paper is on the rightsizing of Cloud deployed clusters, which represent a cost-effective alternative to installation on premises. This paper proposes a novel tool, integrated in a wider DevOps-inspired approach, implementing a parallel and distributed simulation-optimization technique that efficiently and effectively explores the space of alternative Cloud configurations, seeking the minimum cost deployment that satisfies quality of service constraints. The soundness of the proposed solution has been thoroughly validated in a vast experimental campaign encompassing different applications and Big Data platforms.

show abstract

D-SPACE4Cloud: A Design Tool for Big Data Applications

Cited by 9 publications

References 31 publications

Performance Degradation and Cost Impact Evaluation of Privacy Preserving Mechanisms in Big Data Systems

Performance Degradation and Cost Impact Evaluation of Privacy Preserving Mechanisms in Big Data Systems

Portfolio-driven Resource Management for Transient Cloud Servers

Optimizing Quality-Aware Big Data Applications in the Cloud

Contact Info

Product

Resources

About