Enhancing Big Data Application Design with the DICE Framework

Casale, Giuliano; Li, Chen

doi:10.1007/978-3-319-79090-9_13

Cited by 4 publications

(8 citation statements)

References 4 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To our knowledge, only a few non-commercial solutions exist that allow an automated deployment of well-known big data technologies. As briefly described at the beginning of this contribution, these are Apache BigTop, the DICE framework [7], and the Cloudera distribution. While the latter depicts rather a multifunctional suite and provides a variety of wellknown tools, the other two approaches are deployment solutions that allow the provision of targeted tools.…”

Section: Available Non-commercial Deployment Solutions For Big Data T...mentioning

confidence: 99%

“…In contrast to the aforementioned solution, the DICE framework originated from an EU project funded under the Horizon 2020 program, which seeks to "to deliver a quality-driver DevOps toolchain for Big data applications that natively support these Big data technologies" [7]. In doing so, a comprehensive plugin for the integrated development environment (IDE) Eclipse is provided that helps step-by-step with the implementation of data-intensive applications.…”

Section: Available Non-commercial Deployment Solutions For Big Data T...mentioning

confidence: 99%

“…To cover the information required for such a sophisticated artifact, a suitable concept needs to be utilized that delivers an allencompassing overview in terms of existing technologies, their fulfilled functionalities, implementation details as well as relations between each of those. When looking at other approaches, such as the DICE framework [7] or Apache BigTop, this shall offer an opportunity to easily extend or reduce the planned setup to its required technologies.…”

Section: Preliminary Considerationsmentioning

confidence: 99%

“…In contrast to this, free and open-source solutions such as the Developing Data-Intensive Applications with Iterative Quality Enhancements (DICE) framework [7], Apache BigTop, or the Cloudera Hortonworks project, which partially attempt to overcome the referred problems, are in many cases limited in terms of their applicability. For instance, Cloudera offers inter alia their distribution as a sandbox that comes with an extensive collection of big data technologies, however, in many cases they exceed what the user needs, which in turn results in the necessity for a potentially complicated and cumbersome customization and configuration.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Facing Big Data System Architecture Deployments: Towards an Automated Approach Using Container Technologies for Rapid Prototyping

Volk¹,

Staegemann²,

Islam³

et al. 2022

Proceedings of the Annual Hawaii International Conference on System Sciences

View full text Add to dashboard Cite

Within the last decade, big data became a promising trend for many application areas, offering immense potential and a competitive edge for various organizations. As the technical foundation for most of today´s data-intensive projects, not only corresponding infrastructures and facilities but also the appropriate knowledge is required. Currently, several projects and services exist that not only allow enterprises to utilize but also to deploy related technologies and systems. However, at the same time, the use of these is accompanied by various challenges that may result in huge monetary expenditures, a lack of modifiability, or the risk of vendor lock-ins. To overcome these shortcomings, in the contribution at hand, modern container and task automation technologies are used to wrap complex big data technologies into re-usable and portable resources.

show abstract

Section: Available Non-commercial Deployment Solutions For Big Data T...mentioning

confidence: 99%

Section: Available Non-commercial Deployment Solutions For Big Data T...mentioning

confidence: 99%

Section: Preliminary Considerationsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Facing Big Data System Architecture Deployments: Towards an Automated Approach Using Container Technologies for Rapid Prototyping

Volk¹,

Staegemann²,

Islam³

et al. 2022

Proceedings of the Annual Hawaii International Conference on System Sciences

View full text Add to dashboard Cite

show abstract

“…Aliabadi et al [1] present an approach that uses Stochastic Activity Networks for modeling different types of batch applications and how they perform when using different frameworks. In the context of the DICE project [3], methods for modeling Big Data systems have been developed. The models explicitly separate between platform, technology and concrete deployment [5] of a big data application.…”

Section: Related Workmentioning

confidence: 99%

Data Stream Operations as First-Class Entities in Component-Based Performance Models

Werle

Seifermann

Koziolek

2020

Software Architecture

View full text Add to dashboard Cite

Data streaming applications are an important class of dataintensive systems. Performance is an essential quality of such systems. It is, for example, expressed by the delay of analysis results or the utilization of system resources. Architecture-level decisions such as the configuration of sources, sinks and operations, their deployment or the choice of technology impact the performance. Current component-based performance prediction approaches cannot accurately predict the performance of those systems, because they do not support the metrics that are specific to data streaming applications and only approximate the behavior of data stream operations instead of expressing it explicitly. In particular, operations that group multiple data events and thus introduce timing dependencies between different calls to the system are not represented sufficiently. In this paper, we present an approach for modeling networks of data stream operations including their parameters with the goal of predicting the performance of the resulting composed data streaming application. The approach is based on a component-based performance model with queueing semantics for processing resources. Our evaluation shows that our model can more accurately express the behavior of the system, resulting in a more expressive performance model compared to a well-encapsulated component-based model without data stream operations.

show abstract

Accurate Performance Predictions with Component-Based Models of Data Streaming Applications

Werle

Seifermann

Koziolek

2022

Software Architecture

View full text Add to dashboard Cite

Data streaming applications are an important class of dataintensive systems and performance is an essential quality of such systems. Current component-based performance prediction approaches are not sufficient for modeling and predicting the performance of those systems, because the models require elaborate manual engineering to approximate the behavior of data streaming applications that include stateful asynchronous operations, such as windowing operations, and because the simulations for these models do not support the metrics that are specific to data streaming applications. In this paper, we present a modeling language, a simulation and a case-study-based evaluation of the prediction accuracy of an approach for modeling systems that contain stateful asynchronous operations. Our approach directly represents these operations and simulates their behavior. We compare measurements of relevant performance metrics to performance simulation results for a system that processes smart meter readings. To assess the prediction accuracy of our model, we vary both the configuration of the streaming application, such as window sizes, as well as the characteristics of the input data, i.e., the number of smart meters. Our evaluation shows that our model yields prediction results that are competitive with a state-of-the-art baseline model without incurring the additional manual engineering overhead.

show abstract

Enhancing Big Data Application Design with the DICE Framework

Cited by 4 publications

References 4 publications

Facing Big Data System Architecture Deployments: Towards an Automated Approach Using Container Technologies for Rapid Prototyping

Facing Big Data System Architecture Deployments: Towards an Automated Approach Using Container Technologies for Rapid Prototyping

Data Stream Operations as First-Class Entities in Component-Based Performance Models

Accurate Performance Predictions with Component-Based Models of Data Streaming Applications

Contact Info

Product

Resources

About