Query execution for large relations on functional disk system

Nakano, Masaki; Takagi, Mikio

doi:10.1109/icde.1989.47211

Cited by 16 publications

(3 citation statements)

References 7 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…So far we have developed several parallel hardware platforms in addition to the parallel operating systems and the parallel relational algorithms [10,11,16]. The SDC, the super database computer, is one of such platforms and is a cluster of shared memory multiprocessor nodes, and the purpose of the project was to examine the viability of microprocessor based parallel query processing servers.…”

Section: Software Architecture For Query Processingmentioning

confidence: 99%

“…Considerable research on parallel database processing has been done in the last decade [7]. We have developed several parallel hardware platforms in addition to the parallel operating systems and the parallel relational algorithms [10,11,16]. Through the research, the effectiveness of parallelization has been proved, and several vendors are now shipping parallel DBMS products running on commercial parallel platforms.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Parallel database processing on a 100 Node PC cluster

Tamura

Oguchi

1997

Proceedings of the 1997 ACM/IEEE Conference on Supercomputing (CDROM) - Supercomputing '97

View full text Add to dashboard Cite

We developed a PC cluster system consists of 100 PCs. Each PC employs the 200MHz Pentium Pro CPU and is connected with others through an ATM switch. We picked up two kinds of data intensive applications. One is decision support query processing. And the other is data mining, specifically, association rule mining.As a high speed network, ATM technology has recently come to be a de facto standard. While other high performance network standards are also available, ATM networks are widely used from local area to widely distributed environments. One of the problems of the ATM networks is its high latencies, in contrast to their higher bandwidths. This is usually considered a serious flaw of ATM in composing high performance massively parallel processors. However, applications such as large scale database analyses are insensitive to the communication latency, requiring only the bandwidth.On the other hand, the performance of personal computers is increasing rapidly these days while the prices of PCs continue to fall at a much faster rate than workstations'. The 200MHz Pentium Pro CPU is competitive in integer performance to the processor chips found in workstations. Although it is still weak at floating point operations, they are not frequently used in database applications.Thus, by combining PCs and ATM switches we can construct a large scale parallel platform very easily and very inexpensively. In this paper, we examine how such a system can help the data warehouse processing, which currently runs on expensive high-end mainframes and/or workstation servers.In our first experiment, we used the most complex query of the standard benchmark, TPC-D, on a 100 GB database to evaluate the system compared with commercial parallel systems. Our PC cluster exhibited much higher performance compared with those in current TPC benchmark reports. Second, we parallelized association rule mining and ran large scale data mining on the PC cluster. Sufficiently high linearity was obtained. Thus we believe that such commodity based PC clusters will play a very important role in large scale database processing.

show abstract

Section: Software Architecture For Query Processingmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Parallel database processing on a 100 Node PC cluster

Tamura

Oguchi

1997

Proceedings of the 1997 ACM/IEEE Conference on Supercomputing (CDROM) - Supercomputing '97

View full text Add to dashboard Cite

show abstract

“…Several parallel join algorithms have been proposed to alleviate the presence of the data skew; i.e 10,11,7,18,26,14,27,9,20,5,21]. Most of the proposed algorithms are hash-based algorithms.…”

Section: Load Metricsmentioning

confidence: 99%

Skew-insensitive parallel algorithms for relational join

Alsabti

Ranka

Proceedings. Fifth International Conference on High Performance Computing (Cat. No. 98EX238)

View full text Add to dashboard Cite

Join is the most important and expensive operation in relational databases. The parallel join operation is very sensitive to the presence of the data skew. In this paper, we present two new parallel join algorithms for coarse grained machines which work optimally in presence of arbitrary amount of data skew. The rst algorithm is sort-based and the second is hash-based. Both of these algorithms employ a preprocessing phase (prior to the redistribution phase) to equally partition the work among the processors. The proposed algorithms have been designed for memory resident-data. However, they can be extended to disk resident-data. These algorithms are shown to be theoretically as well as practically scalable. Experimental results are provided on the IBM SP-2.

show abstract

A functional disk system for relational database processing: Design and performance evaluation

Nakano

1992

Systems & Computers in Japan

View full text Add to dashboard Cite

This paper considers a relational database processing system and proposes a new high-performance secondary memory system and a functional disk system aiming at the high-speed processing. The traditional commercial relational DBMS employs the simple nest-loop and sortmerge as processing methods for relational algebra operations. However, the proposed functional disk system employs the dynamic clustering algorithm so that the processing load is reduced drastically. Such processes as the hash operation, the cluster management mechanism, the record extraction from the physical block and on-the-fly operation, which takes time by the software, are implemented by hardware. The clustering following the data transfer from the disk is then realized.The performance is improved further by using several processors so that the generated clusters are processed in parallel. Usually, the performance is deteriorated greatly due to the mismatch between 0s and DBMS constructed on 0 s . To solve this problem, the dedicated input/output driver and the buffer management routine were developed in the functional disk system by optimizing the dynamic clustering technique. The functional disk system aims at the drastic performance improvement by introducing the database processing mechanism into the secondary memory system. To verify the effectiveness of this idea, an experimental system was constructed. In spite of the simple structure composed of a disk and 4 MC68020, the experimental system achieved a much higher performance than the existing commercial relational database system in the performance evaluation using the Wisconsin benchmark. Thus, the effectiveness of the functional disk system is verified.

show abstract

Query execution for large relations on functional disk system

Cited by 16 publications

References 7 publications

Parallel database processing on a 100 Node PC cluster

Parallel database processing on a 100 Node PC cluster

Skew-insensitive parallel algorithms for relational join

A functional disk system for relational database processing: Design and performance evaluation

Contact Info

Product

Resources

About