Distribution Rules for Array Database Queries

Abstract

Abstract. Non-trivial retrieval applications involve complex computations on large multi-dimensional datasets. These should, in principle, benefit from the use of relational database technology. However, expressing such problems in terms of relational queries is difficult and timeconsuming. Even more discouraging is the efficiency issue: query optimization strategies successful in classical relational domains may not suffice when applied to the multi-dimensional array domain. The RAM (Relational Array Mapping) system hides these difficulties by providing a transparent mapping between the scientific problem specification and the underlying database system. In addition, its optimizer is specifically tuned to exploit the characteristics of the array paradigm and to allow for automatic balanced work-load distribution. Using an example taken from the multimedia domain, this paper shows how a distributed realword application can be efficiently implemented, using the RAM system, without user intervention.

SciQL, a query language for science applications

“…In the area of building/extending DBMS with array support, the systems can be divided into three groups: i) simulating arrays on top of a relational DBMS, such as RAM [36] and SRAM [9]; ii) storing arrays as BLOBs in relational DBMSs and using an array executor to deal with array specific queries, such as RasDaMan [4]; and iii) enhancing a relational DBMS with array as a primary data type and providing native support for array oriented operations, such as SciQL and SciDB [6,34,10].…”

“…RAM [36] is a proposal for flexible representation of information retrieval models in a single multidimensional array framework. It introduces an array algebra language on top of MonetDB and used as the "gluing layer" for DB+IR applications.…”

Hybrid merge/overlap execution technique for parallel array processing

“…Many engines are being built today to support multidimensional arrays [2,4,5,10,11,21,30]. Many of them supports parallel array processing [2,4,5,10,11,21].…”

“…As a result, many engines have been developed to efficiently process such arrays [2,10,21,30]. Additionally, as arrays grow in size, it is becoming increasingly important to process arrays efficiently in parallel [2,21,27].…”

“…For such operations, two parallel processing techniques are commonly used. In the first technique, which we call merge, an array is spilt into multiple subarrays, each subarray is partly processed in parallel, and the intermediate results are merged [2,4,5,10,11,13,21]. In the second technique, which we call overlap, an array is split into multiple subarrays that overlap by a given number of cells in each dimension.…”