Much ado about shared-nothing

Norman, Michael G.; Zurek, Thomas; Thanisch, Peter

doi:10.1145/234889.234892

Cited by 14 publications

(7 citation statements)

References 6 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Shared-nothing and shared-disk [1] [2] [3] are the two widely-used storage architectures in parallel databases. Both two architectures have their own positive and negative features, but neither of them has the edge on the other in all aspects.…”

Section: Introductionmentioning

confidence: 99%

A Hybrid Shared-Nothing/Shared-Data Storage Scheme for Large-Scale Data Processing

Song

Sun

Chen

2011

2011 IEEE Ninth International Symposium on Parallel and Distributed Processing With Applications

View full text Add to dashboard Cite

Shared-nothing and shared-disk are the two most common storage architectures of parallel databases in the past two decades. Both two types of systems have their own merits for different applications. However, there are no much efforts in investigating the integration of these two architectures and exploiting their merits together. In this paper, we propose a novel hybrid storage architecture for large-scale data processing, to leverage the benefits of both shared-nothing and shared-disk architectures. In the proposed hybrid system, we adopt a sharednothing architecture as the hardware layer and leverage a parallel file system as the storage layer to combine the scattered disks on all database nodes. We present an overall design of the new scheme, including data and storage organization, data access modes, and query processing methods. The proposed hybrid scheme can achieve both high I/O performance as a sharednothing system, and high-speed data sharing across all server nodes as a share-disk system. Preliminary experimental results demonstrate that the hybrid scheme is promising and more appropriate for large-scale and data-intensive applications than each of the two individual types of systems.

show abstract

Section: Introductionmentioning

confidence: 99%

A Hybrid Shared-Nothing/Shared-Data Storage Scheme for Large-Scale Data Processing

Song

Sun

Chen

2011

2011 IEEE Ninth International Symposium on Parallel and Distributed Processing With Applications

View full text Add to dashboard Cite

show abstract

“…The majority of existing parallel database systems work focuses on making use of multiple loosely coupled clusters, typified as shared-nothing systems [5][6][7][8][9][10]. Recently, a new parallel computing trend has emerged.…”

Section: Introductionmentioning

confidence: 99%

Data Parallel Bin-Based Indexing for Answering Queries on Multi-core Architectures

Gosink

Bethel

et al. 2009

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Abstract. The multi-core trend in CPUs and general purpose graphics processing units (GPUs) offers new opportunities for the database community. The increase of cores at exponential rates is likely to affect virtually every server and client in the coming decade, and presents database management systems with a huge, compelling disruption that will radically change how processing is done. This paper presents a new parallel indexing data structure for answering queries that takes full advantage of the increasing thread-level parallelism emerging in multi-core architectures. In our approach, our Data Parallel Bin-based Index Strategy (DP-BIS) first bins the base data, and then partitions and stores the values in each bin as a separate, bin-based data cluster. In answering a query, the procedures for examining the bin numbers and the bin-based data clusters offer the maximum possible level of concurrency; each record is evaluated by a single thread and all threads are processed simultaneously in parallel.We implement and demonstrate the effectiveness of DP-BIS on two multicore architectures: a multi-core CPU and a GPU. The concurrency afforded by DP-BIS allows us to fully utilize the thread-level parallelism provided by each architecture-for example, our GPU-based DP-BIS implementation simultaneously evaluates over 12,000 records with an equivalent number of concurrently executing threads. In comparing DP-BIS's performance across these architectures, we show that the GPU-based DP-BIS implementation requires significantly less computation time to answer a query than the CPU-based implementation. We also demonstrate in our analysis that DP-BIS provides better overall performance than the commonly utilized CPU and GPU-based projection index. Finally, due to data encoding, we show that DP-BIS accesses significantly smaller amounts of data than index strategies that operate solely on a column's base data; this smaller data footprint is critical for parallel processors that possess limited memory resources (e.g. GPUs).

show abstract

“…In shared-nothing architectures, network contention becomes increasingly a bottleneck; in the case of shared-everything, the high degree of resource sharing limits the scaling [Sto86,NZT96].…”

Section: Introductionmentioning

confidence: 99%

Memory Aware Query Routing in Interactive Web-Based Information Systems

Waas

Kersten

2001

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Query throughput is one of the primary optimization goals in interactive web-based information systems in order to achieve the performance necessary to serve large user communities. Queries in this application domain differ significantly from those in traditional database applications: they are of lower complexity and almost exclusively read-only. The architecture we propose here is specifically tailored to take advantage of the query characteristics. It is based on a large parallel shared-nothing database cluster where each node runs a separate server with a fully replicated copy of the database. A query is assigned and entirely executed on one single node avoiding network contention or synchronization effects. However, the actual key to enhanced throughput is a resource efficient scheduling of the arriving queries. We develop a simple and robust scheduling scheme that takes the currently memory resident data at each server into account and trades off memory re-use and execution time, reordering queries as necessary. Our experimental evaluation demonstrates the effectiveness when scaling the system beyond hundreds of nodes showing super-linear speedup.

show abstract

Much ado about shared-nothing

Cited by 14 publications

References 6 publications

A Hybrid Shared-Nothing/Shared-Data Storage Scheme for Large-Scale Data Processing

A Hybrid Shared-Nothing/Shared-Data Storage Scheme for Large-Scale Data Processing

Data Parallel Bin-Based Indexing for Answering Queries on Multi-core Architectures

Memory Aware Query Routing in Interactive Web-Based Information Systems

Contact Info

Product

Resources

About