Differential files

Severance, Dennis G.; Lohman, Guy M.

doi:10.1145/320473.320484

Cited by 253 publications

(46 citation statements)

References 6 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Thus, one potential area for future work is to add the ability to handle incremental inserts to the sample view (assuming that the ACE Tree is most useful in a data warehousing environment, then deletes are far less useful). However, we note that even without the ability to incrementally update an ACE-Tree, it is still easily usable in a dynamic environment if a standard method such as a differential file [27] is applied. Specifically, one could maintain the differential file as a randomly permuted file or even a second ACE Tree, and when a relational selection query is posed, in order to draw a random sample from the query one selects the next sample from either the primary ACE Tree or the differential file with an appropriate hypergeometric probability (for an idea of how this could be done, see the recent paper of Brown and Haas [28] for a discussion of how to draw a single sample from multiple data set partitions).…”

Section: Conclusion and Discussionmentioning

confidence: 99%

Materialized Sample Views for Database Approximation

Joshi

Jermaine

2006

22nd International Conference on Data Engineering (ICDE'06)

View full text Add to dashboard Cite

We consider the problem of creating a sample view of a database table. A sample view is an indexed, materialized view that permits efficient sampling from an arbitrary range query over the view. Such "sample views" are very useful to applications that require random samples from a database: approximate query processing, online aggregation, data mining, and randomized algorithms are a few examples. Our core technical contribution is a new file organization called the ACE Tree that is suitable for organizing and indexing a sample view. One of the most important aspects of the ACE Tree is that it supports online random sampling from the view. That is, at all times, the set of records returned by the ACE Tree constitutes a statistically random sample of the database records satisfying the relational selection predicate over the view. Our paper presents experimental results that demonstrate the utility of the ACE Tree.

show abstract

Section: Conclusion and Discussionmentioning

confidence: 99%

Materialized Sample Views for Database Approximation

Joshi

Jermaine

2006

22nd International Conference on Data Engineering (ICDE'06)

View full text Add to dashboard Cite

show abstract

“…Adaptive merging relies on a form of differential files [38] for high update rates. During a single load operation, multiple new partitions might be created in a partitioned B-tree.…”

Section: Transactions and Partitioned B-treesmentioning

confidence: 99%

Transactional support for adaptive indexing

et al. 2014

View full text Add to dashboard Cite

Adaptive indexing initializes and optimizes indexes incrementally, as a side effect of query processing. The goal is to achieve the benefits of indexes while hiding or minimizing the costs of index creation. However, indexoptimizing side effects seem to turn read-only queries into update transactions that might, for example, create lock contention. This paper studies concurrency control and recovery in the context of adaptive indexing. We show that the design and implementation of adaptive indexing rigorously separates index structures from index contents; this relaxes constraints and requirements during adaptive indexing compared to those of traditional index updates. Our design adapts to the fact that an adaptive index is refined continuously and exploits any concurrency opportunities in a dynamic way. A detailed experimental analysis demonstrates that (a) adaptive indexing maintains its adaptive properties even when running concurrent queries, (b) adaptive indexing can exploit the opportunity for parallelism due to concurrent queries, (c) the number of concurrency conflicts and any concurrency administration overheads follow an adaptive behavior, decreasing as the workload evolves and adapting to the workload needs.

show abstract

“…The idea of collecting updates in a separate space and applying them in a batch was first used in the context of relational databases more than 30 years ago [38]. The idea of that paper was to collect changes in a separate differential file and merge that file regularly with the existing external memory index.…”

Section: Comparison To Differential Filesmentioning

confidence: 99%

“…Several of these optimizations may be traced back to Lars Arge's buffer tree [2]. Graefe also mentions differential files [38] as an effective means to trade query performance for update performance. However, [12] does not mention that one could trade query result staleness and keep both queries and updates efficient as in MOVIES.…”

Section: Extensions For Efficient Updatesmentioning

confidence: 99%

Indexing Moving Objects Using Short-Lived Throwaway Indexes

Dittrich

Blunschi

Salles

2009

Advances in Spatial and Temporal Databases

View full text Add to dashboard Cite

Abstract. With the exponential growth of moving objects data to the Gigabyte range, it has become critical to develop effective techniques for indexing, updating, and querying these massive data sets. To meet the high update rate as well as low query response time requirements of moving object applications, this paper takes a novel approach in moving object indexing. In our approach we do not require a sophisticated index structure that needs to be adjusted for each incoming update. Rather we construct conceptually simple short-lived throwaway indexes which we only keep for a very short period of time (sub-seconds) in main memory. As a consequence, the resulting technique MOVIES supports at the same time high query rates and high update rates and trades this for query result staleness. Moreover, MOVIES is the first main memory method supporting time-parameterized predictive queries. To support this feature we present two algorithms: non-predictive MOVIES and predictive MOVIES. We obtain the surprising result that a predictive indexing approach -considered state-of-the-art in an external-memory scenario -does not scale well in a main memory environment. In fact our results show that MOVIES outperforms state-of-the-art moving object indexes like a main-memory adapted B x -tree by orders of magnitude w.r.t. update rates and query rates. Finally, our experimental evaluation uses a workload unmatched by any previous work. We index the complete road network of Germany consisting of 40,000,000 road segments and 38,000,000 nodes. We scale our workload up to 100,000,000 moving objects, 58,000,000 updates per second and 10,000 queries per second which is unmatched by any previous work.

show abstract

Differential files

Cited by 253 publications

References 6 publications

Materialized Sample Views for Database Approximation

Materialized Sample Views for Database Approximation

Transactional support for adaptive indexing

Indexing Moving Objects Using Short-Lived Throwaway Indexes

Contact Info

Product

Resources

About