Distributing and searching concept hierarchies: an adaptive DHT-based system

Abstract

Concept hierarchies greatly help in the organization and reuse of information and are widely used in a variety of information systems applications. In this paper, we describe a method for efficiently storing and querying data organized into concept hierarchies and dispersed over a DHT. In our method, peers individually decide on the level of indexing according to the granularity of the incoming queries. Roll-up and drill-down operations are performed on a pernode basis in order to minimize the required bandwidth for answering queries on variable aggregation levels. We motivate our approach by applying it on a large-scale Grid system: Specifically, we apply our fully decentralized scheme that creates, queries and updates large volumes of hierarchical data on-line and replace the traditional centralized and strictly indexed information systems. Our extensive experimental results support this argument on many diverse configurations: Our system proves very efficient in skewed workloads, both over single and multiple hierarchy levels at the same time. It adapts to sudden changes in popularity and effectively stores and updates large amounts of data at very low cost.

A distributed tree data structure for real-time OLAP on cloud architectures

“…This is not possible for large data warehouses. For peer-to-peer networks, related work includes distributed methods for querying concept hierarchies such as [25], [26], [27], [28]. However, none of these methods provides real-time OLAP functionality.…”

LinkedPeers: A Distributed System for Interlinking Multidimensional Data

“…The goal is to increase the number of queries answered as exact matches in each dimension. The decision process for a possible re-index is triggered only after an indexed or flooded query only for the reference value, following the procedure described in our previous work [3]. Nevertheless, major enhancements have been made for the customization of the re-indexing operations in multiple dimensions due to the requirements arisen from the interconnection among the rings.…”

Parallel Olap on Multi/Many-Core and Cloud Platforms

“…Asiki et al in [17] presented a similar method that uses concept hierarchies; however, similar to [43], it does not contain a centralized tree structure. In another study by Asiki et al in [18,19], authors improved their previous work and implemented a decentralized concept hierarchy on a grid system. The new method is similar to the DC-tree and is able to perform updates and lookups simultaneously.…”

“…In another study by Doka et al [45], an online system was presented that is capable of performing online OLAP operations. Their method is based on concept hierarchies similar to [17,18,19]; however, the method lacks a multi-dimensional data indexing structure such as R-tree. The method tries to integrate the concept hierarchies of different dimensions and use them to store input data.…”

“…In spite of the recent efforts, parallelization of Real-time OLAP on multi/many-core processors has not been addressed yet. However, as referred above, there have been few methods ( [18,19,45,44] Apart from data warehousing methods, there have been a few studies for providing indexing structures on cloud systems. These efforts were usually for general data indexing in 1-dimensional and multi-dimensional spaces.…”