Revisiting reuse for approximate query processing

Galakatos, Alex; Crotty, Andrew; Zgraggen, Emanuel; Binnig, Carsten; Kraska, Tim

doi:10.14778/3115404.3115418

Cited by 62 publications

(57 citation statements)

References 37 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Approximate query engines show promise for interactive use cases [22], however these systems have not been designed with type1 latency in mind. In particular, these systems assume they are being executed in a programming environment (e.g., Jupyter Notebook, etc.…”

Section: Optimization Opportunitiesmentioning

confidence: 99%

Database Benchmarking for Supporting Real-Time Interactive Querying of Large Data

Battle

Eichmann

Angelini

et al. 2020

Proceedings of the 2020 ACM SIGMOD International Conference on Management of Data

Self Cite

View full text Add to dashboard Cite

In this paper, we present a new benchmark to validate the suitability of database systems for interactive visualization workloads. While there exist proposals for evaluating database systems on interactive data exploration workloads, none rely on real user traces for database benchmarking. To this end, our long term goal is to collect user traces that represent workloads with different exploration characteristics. In this paper, we present an initial benchmark that focuses on "crossfilter"-style applications, which are a popular interaction type for data exploration and a particularly demanding scenario for testing database system performance. We make our benchmark materials, including input datasets, interaction sequences, corresponding SQL queries, and analysis code, freely available as a community resource, to foster further research in this area: https://osf.io/9xerb/?view_only= 81de1a3f99d04529b6b173a3bd5b4d23. CCS CONCEPTS • Information systems → Data management systems; Data analytics; • Human-centered computing → Visualization systems and tools.

show abstract

Section: Optimization Opportunitiesmentioning

confidence: 99%

Database Benchmarking for Supporting Real-Time Interactive Querying of Large Data

Battle

Eichmann

Angelini

et al. 2020

Proceedings of the 2020 ACM SIGMOD International Conference on Management of Data

Self Cite

View full text Add to dashboard Cite

show abstract

“…Incremental AQP Galakatos et al [42] propose an AQP formulation that treats aggregate query answers as random variables to enable reusing of approximate results with reasoning about error propagation across overlapping queries. When a new query is coming, it finds previous queries which have common attributes and query conditions with the query, thus uses these results to refine the approximation.…”

Section: Other Workmentioning

confidence: 99%

Approximate Query Processing: What is New and Where to Go?

2018

Data Sci. Eng.

112

View full text Add to dashboard Cite

Online analytical processing (OLAP) is a core functionality in database systems. The performance of OLAP is crucial to make online decisions in many applications. However, it is rather costly to support OLAP on large datasets, especially big data, and the methods that compute exact answers cannot meet the high-performance requirement. To alleviate this problem, approximate query processing (AQP) has been proposed, which aims to find an approximate answer as close as to the exact answer efficiently. Existing AQP techniques can be broadly categorized into two categories. (1) Online aggregation: select samples online and use these samples to answer OLAP queries. (2) Offline synopses generation: generate synopses offline based on a-priori knowledge (e.g., data statistics or query workload) and use these synopses to answer OLAP queries. We discuss the research challenges in AQP and summarize existing techniques to address these challenges. In addition, we review how to use AQP to support other complex data types, e.g., spatial data and trajectory data, and support other applications, e.g., data visualization and data cleaning. We also introduce existing AQP systems and summarize their advantages and limitations. Lastly, we provide research challenges and opportunities of AQP. We believe that the survey can help the partitioners to understand existing AQP techniques and select appropriate methods in their applications.

show abstract

“…ReCache studies the same problem for heterogeneous data sources [8]. Intermediate results can also accelerate approximate query processing [26] and feature selection work-loads [63]. IQP considers how to efficiently incorporate delta into an existing query result, rather than storing materialized views or intermediate states for future queries.…”

Section: Incremental View Maintenancementioning

confidence: 99%

Intermittent query processing

et al. 2019

View full text Add to dashboard Cite

Many applications ingest data in an intermittent, yet largely predictable, pattern. Existing systems tend to ignore how data arrives when making decisions about how to update (or refresh) an ongoing query. To address this shortcoming we propose a new query processing paradigm, Intermittent Query Processing (IQP), that bridges query execution and policies, to determine when to update results and how much resources to allocate for ensuring fast query updates. Here, for a query the system provides an initial result that is to be refreshed when policy dictates, such as after a defined number of new records arrive or a time interval elapses. In between intermittent data arrivals, IQP inactivates query execution by selectively releasing some resources occupied in normal execution that will be least helpful (for future refreshes) according to the arrival patterns for new records. We present an IQP prototype based on PostgreSQL that selectively persists the state associated with query operators to allow for fast query updates while constraining resource consumption. Our experiments show that for several application scenarios IQP greatly lowers query processing latency compared to batch systems, and largely reduces memory consumption with comparable latency compared to a state-of-theart incremental view maintenance technique.

show abstract

Revisiting reuse for approximate query processing

Cited by 62 publications

References 37 publications

Database Benchmarking for Supporting Real-Time Interactive Querying of Large Data

Database Benchmarking for Supporting Real-Time Interactive Querying of Large Data

Approximate Query Processing: What is New and Where to Go?

Intermittent query processing

Contact Info

Product

Resources

About