Brown Dwarf: A fully-distributed, fault-tolerant data warehousing system

Abstract

a b s t r a c tIn this paper we present the Brown Dwarf, a distributed data analytics system designed to efficiently store, query and update multidimensional data over commodity network nodes, without the use of any proprietary tool. Brown Dwarf distributes a centralized indexing structure among peers on-the-fly, reducing cube creation and querying times by enforcing parallelization. Analytical queries are naturally performed on-line through cooperating nodes that form an unstructured Peer-to-Peer overlay. Updates are also performed on-line, eliminating the usually costly over-night process. Moreover, the system employs an adaptive replication scheme that adjusts to the workload skew as well as the network churn by expanding or shrinking the units of the distributed data structure. Our system has been thoroughly evaluated on an actual testbed: it manages to accelerate cube creation up and querying up to several tens of times compared to the centralized solution by exploiting the capabilities of the available network nodes working in parallel. It also manages to quickly adapt even after sudden bursts in load and remains unaffected with a considerable fraction of frequent node failures. These advantages are even more apparent for dense and skewed data cubes and workloads.

A distributed tree data structure for real-time OLAP on cloud architectures

“…This is not possible for large data warehouses. For peer-to-peer networks, related work includes distributed methods for querying concept hierarchies such as [25], [26], [27], [28]. However, none of these methods provides real-time OLAP functionality.…”

Full and partial data cube computation and representation over commodity PCs

“…Furthermore, the cube construction is still sequential. Finally, the network communication becomes a bottleneck when massive cubes are computed [4].…”

“…Among the distributed approaches, we can point out RP (Replicated Parallel BUC), BPP (Breadth-first writing, Partitioned, Parallel-BUC), ASL (Affinity Sip List), and PT (Partitioned Tree), presented in [13]. Besides these approaches, we have "Pipe 'n Prune" (PnP) approach [2] and Brown Dwarf approach [4] among the most promising alternatives the literature. PnP approach is the unique approach that achieves linear speedup in computing partial data cubes over distributed memory architectures.…”

Parallel Olap on Multi/Many-Core and Cloud Platforms

“…This is in contrast to the DC-tree method in which input data is stored in a single tree-like data structure. The most recent work by Doka et al [44] called Brown Dwarf which tries to build a distributed version of the originally centralized Dwarf method [106] in peer-to-peer networks. The method is parallelized over the network and provides online update and query transactions.…”

“…In spite of the recent efforts, parallelization of Real-time OLAP on multi/many-core processors has not been addressed yet. However, as referred above, there have been few methods ( [18,19,45,44] Apart from data warehousing methods, there have been a few studies for providing indexing structures on cloud systems. These efforts were usually for general data indexing in 1-dimensional and multi-dimensional spaces.…”