ALOJA: A systematic study of Hadoop deployment variables to enable automated characterization of cost-effectiveness

Abstract

Abstract-This article presents the ALOJA project, an initiative to produce mechanisms for an automated characterization of cost-effectiveness of Hadoop deployments and reports its initial results. ALOJA is the latest phase of a long-term collaborative engagement between BSC and Microsoft which, over the past 6 years has explored a range of different aspects of computing systems, software technologies and performance profiling. While during the last 5 years, Hadoop has become the de-facto platform for Big Data deployments, still little is understood of how the different layers of the software and hardware deployment options affects its performance. Early ALOJA results show that Hadoop's runtime performance, and therefore its price, are critically affected by relatively simple software and hardware configuration choices e.g., number of mappers, compression, or volume configuration. Project ALOJA presents a vendor-neutral repository featuring over 5000 Hadoop runs, a test bed, and tools to evaluate the cost-effectiveness of different hardware, parameter tuning, and Cloud services for Hadoop. As few organizations have the time or performance profiling expertise, we expect our growing repository will benefit Hadoop customers to meet their Big Data application needs. ALOJA seeks to provide both knowledge and an online service to with which users make better informed configuration choices for their Hadoop compute infrastructure whether this be on-premise or cloud-based.The initial version of ALOJA's Web application and sources are available at http://hadoop.bsc.es

ALOJA: A Benchmarking and Predictive Platform for Big Data Performance Analysis

“…Over time, collecting performance metrics have also become a Big Data problem: we have over 1.2TB of performance metrics for the executions after importing the executions into the database. A description of the architecture in ALOJA can be found in [11]. While the data is 45x smaller than traces from profiling as can be seen in Figure 2, as we get more executions, we currently use this data mostly for debugging executions manually.…”

“…While the results from the data aggregation efforts allows to process data interactively for the analytic online tools [11], the increasing number of configuration choices as the project expands in architectures and services -in the millions for benchmarks that a single iteration can take hours to execute. In order to cope with the increasing number of configuration options the project was faced with the need first to do manual sampling from the search space, and grouping of results to extrapolate results between clusters.…”

“…During the development of the defined phases for project ALOJA [11], we have experienced a shift from an initial approach of using low-level HPC tools to profile Hadoop runtime [3] based BSC's previous expertise to higher-level performance analysis. Part of the initial work included inserting hooks into the Hadoop source code to capture application events, that are later post-processed into the format of BSC's HPC tools, which are used to analyze the performance and parallel efficiency of supercomputing or MPI-based workloads.…”

“…The ALOJA project is a research initiative from the Barcelona Supercomputing Center (BSC) with support from Microsoft Research and product groups [11] to explore and produce a systematic study of Hadoop configuration and deployment options. The study includes the main software configurations that can greatly impact Hadoop's performance [7]; as well as different hardware choices to evaluating their effectiveness and use cases including: current commodity hardware on which Hadoop systems where originally designed for [1], low-power devices, high-end servers, to new storage and networking 2 ; as well as new managed Cloud services (PaaS).…”

“…We use the Intel HiBench benchmark suite [19] as representative workloads, and we use the ALOJA-ML toolkit, the machine learning extension of the ALOJA framework [21] for such learning. The purpose of ALOJA-ML is to discover knowledge from Hadoop environments, first predicting execution times for given known workloads depending on the Hadoop configuration and the provided hardware resources; then evaluating which elements of a given deployment are the most relevant to reduce such running times.…”