Correlation-based performance analysis for full-system MapReduce optimization

Context-aware data quality assessment for big data

“…In [25], the application non functional requirements are considered by proposing a new cloud service for scaling the infrastructure to meet them. Performances are also considered in [26], where the authors propose a correlation-based performance analysis to identify critical outliers by correlating different phases, tasks, and resources.…”

HybridTune: Spatio-temporal Data and Model Driven Performance Diagnosis for Big Data Systems

“…applications. SONATA [16] propose a correlation-based performance analysis approach for full-system MapReduce optimization, it correlates different phases, tasks and resources for identify critical outliers and recommends optimization suggestions based on embedded rules, which just uses the model-based method. HiTune [10] describe a dataflowdriven performance analysis approach, it reconstruct the high level, dataflow-based, distributed and dynamic execution process for each Big Data application.…”

“…Because Big Data systems are likely to be constructed from thousands of distributed computing machines, this means that performance issues may exist in a wide variety of subsets or heterogeneous node configurations, such as processors, memory, disks and network; Moreover, the entire software/hardware stacks of Big Data applications are also very complicated and include hundreds of adjustable parameters, which make performance analysis is more complex and needs fine-grained performance data collection and multi-level data association. So far the majority of state-of-the-art performance optimization approaches of Big Data Systems has focused on performance analysis [10] [16] [15] [32] [9] and Big Data systems tuning [18], etc,. Although existing studies pay much efforts on overall performance analysis of Big Data Systems in particular, and have made substantial progress.…”