Multi-Query Optimization Revisited: A Full-Query Algebraic Method

Tu, Yi-Cheng; Eslami, Mehrad; Xu, Zichen; Charkhgard, Hadi

doi:10.1109/bigdata55660.2022.10020338

Search citation statements

Order By: Relevance

Paper Sections

Select...

Citation Types

Supporting

Mentioning

Contrasting

Year Published

2023

2024

Publication Types

Select...

Other1

Article1

Relationship

Self Cite0

Independent2

Authors

Journals

Cited by 2 publications

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

Batch Hop-Constrained s-t Simple Path Query Processing in Large Graphs

Yuan,

Hao,

Lin

et al. 2024

2024 IEEE 40th International Conference on Data Engineering (ICDE)

View full text Add to dashboard Cite

Batch Hop-Constrained s-t Simple Path Query Processing in Large Graphs

Yuan,

Hao,

Lin

et al. 2024

2024 IEEE 40th International Conference on Data Engineering (ICDE)

View full text Add to dashboard Cite

GEqO: ML-Accelerated Semantic Equivalence Detection

Haynes,

Alotaibi,

Pavlenko

et al. 2023

Proc. ACM Manag. Data

View full text Add to dashboard Cite

Large scale analytics engines have become a core dependency for modern data-driven enterprises to derive business insights and drive actions. These engines support a large number of analytic jobs processing huge volumes of data on a daily basis, and workloads are often inundated with overlapping computations across multiple jobs. Reusing common computation is crucial for efficient cluster resource utilization and reducing job execution time. Detecting common computation is the first and key step for reducing this computational redundancy. However, detecting equivalence on large-scale analytics engines requires efficient and scalable solutions that are fully automated. In addition, to maximize computation reuse, equivalence needs to be detected at the semantic level instead of just the syntactic level (i.e., the ability to detect semantic equivalence of seemingly different-looking queries). Unfortunately, existing solutions fall short of satisfying these requirements. In this paper, we take a major step towards filling this gap by proposing GEqO, a portable and lightweight machine-learning-based framework for efficiently identifying semantically equivalent computations at scale. GEqO introduces two machine-learning-based filters that quickly prune out nonequivalent subexpressions and employs a semi-supervised learning feedback loop to iteratively improve its model with an intelligent sampling mechanism. Further, with its novel database-agnostic featurization method, GEqO can transfer the learning from one workload and database to another. Our extensive empirical evaluation shows that, on TPC-DS-like queries, GEqO yields significant performance gains-up to 200x faster than automated verifiers-and finds up to 2x more equivalences than optimizer and signature-based equivalence detection approaches.

show abstract

Multi-Query Optimization Revisited: A Full-Query Algebraic Method

Cited by 2 publications

References 20 publications

Batch Hop-Constrained s-t Simple Path Query Processing in Large Graphs

Batch Hop-Constrained s-t Simple Path Query Processing in Large Graphs

GEqO: ML-Accelerated Semantic Equivalence Detection

Contact Info

Product

Resources

About