On-Line Index Selection for Shifting Workloads

Schnaitter, Karl; Abiteboul, Serge; Milo, Tova; Polyzotis, Neoklis

doi:10.1109/icdew.2007.4401029

Cited by 58 publications

(35 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Both offline [1], [8], [9] and online [10], [11], [12] tools find an optimal set of indexes within user specified constraints (e.g., maximum index size). Rather than replicate this work, Shinobi analyzes the output of such tools (or hand-crafted physical designs), and runs index selection and partitioning techniques to further optimize their designs by identifying subsets of a table where installing an index will be detrimental to performance.…”

Section: Related Workmentioning

confidence: 99%

Partitioning techniques for fine-grained indexing

Wu¹,

Madden²

2011

2011 IEEE 27th International Conference on Data Engineering

View full text Add to dashboard Cite

Abstract-Many data-intensive websites use databases that grow much faster than the rate that users access the data. Such growing datasets lead to ever-increasing space and performance overheads for maintaining and accessing indexes. Furthermore, there is often considerable skew with popular users and recent data accessed much more frequently. These observations led us to design Shinobi, a system which uses horizontal partitioning as a mechanism for improving query performance to cluster the physical data, and increasing insert performance by only indexing data that is frequently accessed. We present database design algorithms that optimally partition tables, drop indexes from partitions that are infrequently queried, and maintain these partitions as workloads change. We show a 60× performance improvement over traditionally indexed tables using a real-world query workload derived from a traffic monitoring application

show abstract

Section: Related Workmentioning

confidence: 99%

Partitioning techniques for fine-grained indexing

Wu¹,

Madden²

2011

2011 IEEE 27th International Conference on Data Engineering

View full text Add to dashboard Cite

show abstract

“…Though, this works quite well for static scenarios with fixed workloads, dynamic environments (in terms of data characteristics and query workload) still pose a serious challenge. One possible solution is to monitor the workload and adapt the configuration continuously [1], [2], [3]. However, changing a configuration at runtime, e.g.…”

Section: Introductionmentioning

confidence: 99%

Automatic schema design for co-clustered tables

Baumann

Boncz

Sattler

2013

2013 IEEE 29th International Conference on Data Engineering Workshops (ICDEW)

View full text Add to dashboard Cite

Abstract-Schema design of analytical workloads provides opportunities to index, cluster, partition and/or materialize. With these opportunities also the complexity of finding the right setup rises. In this paper we present an automatic schema design approach for a table co-clustering scheme called Bitwise Dimensional Co-Clustering, aimed at schemas with a moderate number of dimensions, but not limited to typical star and snowflake schemas. The goal is to design one primary schema and keep the knobs to turn to a minimum while providing a robust schema for a wide range of queries. In our approach a clustered schema is derived by trying to apply dimensions throughout the whole schema and co-cluster as many tables as possible according to at least one common dimension. Our approach is based on the assumption that initially foreign key relationships and a set of dimensions are defined based on classic DDL.

show abstract

“…Continuous On-Line Tuning (COLT) discussed by Schnaitter et al (2006Schnaitter et al ( , 2007 is a framework that continuously examines the workload and proposes the indexes to make the physical design more valuable. COLT performs the index selection in three stages, i.e.…”

Section: Self-configurationmentioning

confidence: 99%

Workload management: a technology perspective with respect to self-* characteristics

et al. 2012

View full text Add to dashboard Cite

Rapid growth in data, maximum functionality requirements and changing behavior in the database workload tends the workload management to be more complex. Organizations have complex type of workloads that are very difficult to manage by humans and even in some cases this management becomes impossible. Human experts take long time to get sufficient experience so that they can manage the workload efficiently. The versatility in workload due to huge data size and user requirements leads us towards the new challenges. One of the challenges is the identification of the problematic queries and the decision about these, i.e. whether to continue their execution or stop. The other challenge is how to characterize the workload, as the tasks such as configuration, prediction and adoption are fully dependent on the workload characterization. Correct and timely characterization leads managing the workload in an efficient manner and vice versa. In this scenario, our objective is to produce a workload management strategy or framework that is fully adoptive. The paper provides a summary of the structure and achievements of the database tools that exhibit Autonomic Computing or self-* characteristics in workload management. We have categorized the database workload tools to these self-* characteristics and identified their limitations. Finally the paper presents the research done in the database workload management tools with respect to the workload type and Autonomic Computing.

show abstract

On-Line Index Selection for Shifting Workloads

Cited by 58 publications

References 19 publications

Partitioning techniques for fine-grained indexing

Partitioning techniques for fine-grained indexing

Automatic schema design for co-clustered tables

Workload management: a technology perspective with respect to self-* characteristics

Contact Info

Product

Resources

About