A query processing strategy for the decomposed storage model

Khoshafian, Setrag; Copeland, George P.; Jagodits, Thomas; Boral, Haran; Valduriez, Patrick

doi:10.1109/icde.1987.7272433

Cited by 42 publications

(25 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…In this paper, we revisit this literature on compression in the context of column-oriented database systems [28,9,10,18,21,1,19]. A column-oriented database system (or "column-store") is one in which each attribute is stored in a separate column, such that successive values of that attribute are stored consecutively on disk.…”

Section: Introductionmentioning

confidence: 99%

Integrating compression and execution in column-oriented database systems

Abadi

Madden

Ferreira

2006

Proceedings of the 2006 ACM SIGMOD International Conference on Management of Data

453

360

View full text Add to dashboard Cite

Column-oriented database system architectures invite a reevaluation of how and when data in databases is compressed. Storing data in a column-oriented fashion greatly increases the similarity of adjacent records on disk and thus opportunities for compression. The ability to compress many adjacent tuples at once lowers the per-tuple cost of compression, both in terms of CPU and space overheads.In this paper, we discuss how we extended C-Store (a column-oriented DBMS) with a compression sub-system. We show how compression schemes not traditionally used in roworiented DBMSs can be applied to column-oriented systems. We then evaluate a set of compression schemes and show that the best scheme depends not only on the properties of the data but also on the nature of the query workload.

show abstract

Section: Introductionmentioning

confidence: 99%

Integrating compression and execution in column-oriented database systems

Abadi

Madden

Ferreira

2006

Proceedings of the 2006 ACM SIGMOD International Conference on Management of Data

453

360

View full text Add to dashboard Cite

show abstract

“…The practical conclusions from this work, reported in [VKC86] and cited in [KCJ+87], is (1) that DSM with join indexes provides better retrieval performance than NSM when the number of retrieved attributes is low or the number of retrieved records is medium to high, while NSM provides better retrieval performance when the number of retrieved attributes is high and the number of retrieved records is low, and (2) that the performance of single attribute modification is the same for both DSM and NSM, while NSM provides better record insert/delete performance. This is an approach which is similar to those used in MonetDB [MBK00a,MBK00b] and in Cantor [Sve82], with the following main differences: (1) DSM provides two predefined join indices for each attribute, one clustered on each of the two attributes (attribute value, TID), while Cantor and MonetDB both use indices which are created as needed during query evaluation; (2) Cantor stores these indices using Run-Length-Encoding (RLE) compression; MonetDB introduces a novel radix cluster algorithm for hash join; (3) although potentially important, parallelism has not been presented as a key design issue for MonetDB, nor was it one for Cantor; (4) the algorithms used in MonetDB and Cantor were both presented as simple two-way joins, corresponding mainly to the composition phase in the DSM algorithm, which is presented as an m-way join.…”

Section: Transposed Files and The Decomposed Storage Modelmentioning

confidence: 99%

“…While the authors of [CK85] [KCJ+87], the DSM further assumes that two copies of each binary relation are stored, one clustered (i.e., sorted or hashed with an index) on each of the two attributes (TID, value). The authors of [CK85] conclude that there seems to be a general consensus among the database community that the conventional N-ary Storage Model (NSM) is better.…”

Section: Transposed Files and The Decomposed Storage Modelmentioning

confidence: 99%

See 1 more Smart Citation

Emerging Database Systems in Support of Scientific Data

Svensson¹,

Boncz²,

Ivanova³

et al. 2009

Scientific Data Management

View full text Add to dashboard Cite

Abstract. This chapter surveys and discusses the evolution of a certain class of database architectures, more recently referred to as "vertical databases". The topics discussed in this chapter include the evolution of storage structures from the 1970"s till now, data compression techniques, and query processing techniques for single-and multi-variable queries in vertical databases. Next, the chapter covers in detail the architecture and design considerations of a particular (open source) vertical database system, called MonetDB. This is followed by an example of using MonetDB for the SkyServer data, and the query processing improvements it offers.

show abstract

“…Recommending a physical configuration for a given workload has been widely studied since 1987 [16]. Most of the existing works [17][18][19][20][21][22][23] propose effective methods to estimate the cost of a given workload over candidate physical configurations.…”

Section: Related Workmentioning

confidence: 99%

Diverse Mobile System for Location‐Based Mobile Data

Liao

Tan

Luo

et al. 2018

Wireless Communications and Mobile Computing

View full text Add to dashboard Cite

The value of large amount of location-based mobile data has received wide attention in many research fields including human behavior analysis, urban transportation planning, and various location-based services. Nowadays, both scientific and industrial communities are encouraged to collect as much location-based mobile data as possible, which brings two challenges: (1) how to efficiently process the queries of big location-based mobile data and (2) how to reduce the cost of storage services, because it is too expensive to store several exact data replicas for fault-tolerance. So far, several dedicated storage systems have been proposed to address these issues. However, they do not work well when the ranges of queries vary widely. In this work, we design a storage system based on diverse replica scheme which not only can improve the query processing efficiency but also can reduce the cost of storage space. To the best of our knowledge, this is the first work to investigate the data storage and processing in the context of big location-based mobile data. Specifically, we conduct in-depth theoretical and empirical analysis of the trade-offs between different spatial-temporal partitioning and data encoding schemes. Moreover, we propose an effective approach to select an appropriate set of diverse replicas, which is optimized for the expected query loads while conforming to the given storage space budget. The experiment results show that using diverse replicas can significantly improve the overall query performance and the proposed algorithms for the replica selection problem are both effective and efficient.

show abstract

A query processing strategy for the decomposed storage model

Cited by 42 publications

References 9 publications

Integrating compression and execution in column-oriented database systems

Integrating compression and execution in column-oriented database systems

Emerging Database Systems in Support of Scientific Data

Diverse Mobile System for Location‐Based Mobile Data

Contact Info

Product

Resources

About