Dynamic Histograms: Capturing Evolving Data Sets

Donjerkovic, D.; Ioannidis, Yannis; Ramakrishnan, Raghu

doi:10.1109/icde.2000.839394

Cited by 40 publications

(37 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The histogram-based technique we use is influenced from previous work on maintaining dynamic histograms on data distributions for accurately estimating the selectivity of query predicates [14,15]. In our case, we are interested in the frequency of accesses to a particular region, rather than the data distribution, and on the access pattern.…”

Section: Dynamic Load Balancingmentioning

confidence: 99%

Scalable and dynamically balanced shared-everything OLTP with physiological partitioning

et al. 2012

View full text Add to dashboard Cite

Scaling the performance of shared-everything transaction processing systems to highly parallel multicore hardware remains a challenge for database system designers. Recent proposals alleviate locking and logging bottlenecks in the system, leaving page latching as the next potential problem. To tackle the page latching problem, we propose physiological partitioning (PLP). PLP applies logical-only partitioning, maintaining the desired properties of sharedeverything designs, and introduces a multi-rooted B+Tree index structure (MRBTree) that enables the partitioning of the accesses at the physical page level. Logical partitioning and MRBTrees together ensure that all accesses to a given index page come from a single thread and, hence, can be entirely latch free; an extended design makes heap page accesses thread private as well. Moreover, MRBTrees offer an infrastructure for easy repartitioning and allow us to have a lightweight dynamic load balancing mechanism (DLB) on top of PLP. Profiling a PLP prototype running on different multicore machines shows that it acquires 85 and 68 % fewer contentious critical sections, respectively, than an optimized conventional design and one based on logical-only partitioning. PLP also improves performance up to almost 50 % over the existing systems, while DLB enhances the system with rapid and robust behavior in both detecting and handling load imbalances.

show abstract

Section: Dynamic Load Balancingmentioning

confidence: 99%

Scalable and dynamically balanced shared-everything OLTP with physiological partitioning

et al. 2012

View full text Add to dashboard Cite

show abstract

“…Another class of histograms is dynamic histograms [12,18], that are maintained incrementally and therefore better suited for our approach. Most histograms described in the literature are equi-depth histograms, since these capture distributions better than equi-width histograms for the same number of buckets [18].…”

Section: Histogram Designmentioning

confidence: 99%

DYFRAM: dynamic fragmentation and replica management in distributed database systems

Hauglid

Ryeng

Nørvåg

2010

Distrib Parallel Databases

View full text Add to dashboard Cite

In distributed database systems, tables are frequently fragmented and replicated over a number of sites in order to reduce network communication costs. How to fragment, when to replicate and how to allocate the fragments to the sites are challenging problems that has previously been solved either by static fragmentation, replication and allocation, or based on a priori query analysis. Many emerging applications of distributed database systems generate very dynamic workloads with frequent changes in access patterns from different sites. In such contexts, continuous refragmentation and reallocation can significantly improve performance. In this paper we present DYFRAM, a decentralized approach for dynamic table fragmentation and allocation in distributed database systems based on observation of the access patterns of sites to tables. The approach performs fragmentation, replication, and reallocation based on recent access history, aiming at maximizing the number of local accesses compared to accesses from remote sites. We show through simulations and experiments on the DASCOSA distributed database system that the approach significantly reduces communication costs for typical access patterns, thus demonstrating the feasibility of our approach.

show abstract

“…In this experiment, we use two techniques to collect information: incremental histograms (using the Dynamic Compressed histograms of [7]) and order detection. (Uniqueness can also be quickly detected in the special case where the values are sorted.…”

Section: Evidence That Selectivity Is Predictablementioning

confidence: 99%

Adapting to source properties in processing data integration queries

Ives

Halevy

Weld

2004

Proceedings of the 2004 ACM SIGMOD International Conference on Management of Data

View full text Add to dashboard Cite

An effective query optimizer finds a query plan that exploits the characteristics of the source data. In data integration, little is known in advance about sources' properties, which necessitates the use of adaptive query processing techniques to adjust query processing on-the-fly. Prior work in adaptive query processing has focused on compensating for delays and adjusting for mis-estimated cardinality or selectivity values. In this paper, we present a generalized architecture for adaptive query processing and introduce a new technique, called adaptive data partitioning (ADP), which is based on the idea of dividing the source data into regions, each executed by different, complementary plans. We show how this model can be applied in novel ways to not only correct for underestimated selectivity and cardinality values, but also to discover and exploit order in the source data, and to detect and exploit source data that can be effectively pre-aggregated. We experimentally compare a number of alternative strategies and show that our approach is effective.

show abstract

Dynamic Histograms: Capturing Evolving Data Sets

Cited by 40 publications

References 10 publications

Scalable and dynamically balanced shared-everything OLTP with physiological partitioning

Scalable and dynamically balanced shared-everything OLTP with physiological partitioning

DYFRAM: dynamic fragmentation and replica management in distributed database systems

Adapting to source properties in processing data integration queries

Contact Info

Product

Resources

About