CHAC: An Effective Attribute Clustering Algorithm for Large-Scale Data Processing

Abstract

Nowadays Hadoop has become a leading architecture for large-scale data processing. One of the efficient ways to accelerate data processing is column-oriented storage technique which has been integrated into Hadoop family recently. However, how to design an appropriate attribute clustering algorithm to achieve optimal data processing performance for columnoriented hadoop system is still a big problem. In this paper, we propose a novel algorithm called CHAC to solve this problem. Both cases of overlapping attribute cluster and non-overlapping attribute cluster are considered in CHAC. In addition, an adjustable parameter is also taken into account to prohibit excessive attribute redundancy via limiting space overhead. The experimental results on TPC-H Benchmark demonstrate the efficiency and effectiveness of the proposed algorithm.

Application of Dynamic Fragmentation Methods in Multimedia Databases: A Review

“…Figure 5 shows the number of proposals related to each type of fragmentation. Articles [ 1 , 2 , 3 , 38 , 39 , 40 , 41 , 42 , 43 , 44 , 45 , 46 , 47 , 48 , 49 , 50 , 51 , 52 , 53 , 54 , 55 , 56 , 57 , 58 , 59 , 60 , 61 , 62 , 63 , 64 , 65 , 66 ] focus on horizontal fragmentation; while [ 7 , 22 , 23 , 24 , 25 , 26 , 27 , 28 , 29 , 30 , 31 , 32 , 33 , 34 , 35 , 36 , 37 ] consider vertical partitioning, [ 67 , 68 , 69 , 70 , 71 , 72 ,…”

“…The most relevant work in the category “Fragmentation of Multimedia Databases” is [ 1 ] since it has been cited in several papers ([ 26 , 32 , 40 , 41 , 55 , 67 , 73 ]); in contrast, the most influential method in the category “Dynamic Fragmentation” is [ 70 ] because it has motivated a number of approaches ([ 28 , 29 , 30 , 31 , 32 , 35 , 52 , 56 , 60 , 66 , 69 , 70 , 75 , 77 , 79 , 91 ]); in the category “Fragmentation for NOSQL DBMS”, the article that has attracted more attention is [ 63 ]; finally, [ 25 ] is the most cited work in the category “Other Types of Fragmentation” ([ 23 , 26 , 27 , 32 ]).…”

“…A new algorithm called CHAC (Column-oriented Hadoop based Attribute Clustering) was proposed in [ 27 ] to design an appropriate attribute grouping algorithm to achieve optimal data processing performance in the column-oriented Hadoop environment. To perform the tests, the TPC-H benchmark was used and the algorithm was evaluated in 16 nodes of which one of them acted as the master.…”