Query optimization Approach with Shuffle Intermediate Cache Layer for Spark SQL

Zhai, Mingyu; Song, Aibo; Qiu, Jingyi; Ji, Xuechun; Wu, Qingxi

doi:10.1109/ipccc47392.2019.8958719

Search citation statements

Order By: Relevance

Paper Sections

Select...

Introduction1

Citation Types

Supporting

Mentioning

Contrasting

Year Published

2021

Publication Types

Select...

Other2

Relationship

Self Cite0

Independent2

Authors

Journals

Cited by 2 publications

(1 citation statement)

References 9 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Previous work optimising SQL queries has mostly concerned optimisations on the SQL server side, including optimising the execution schedule [5], row matching algorithm [6], compilation changes [7], and changes to server implementation [8]. These optimisations aim to be generic to many queries rather than finding optimisations for specific queries.…”

Section: Introductionmentioning

confidence: 99%

Optimising SQL Queries Using Genetic Improvement

Callan

Petke

2021

2021 IEEE/ACM International Workshop on Genetic Improvement (GI)

View full text Add to dashboard Cite

Structured Query Language (SQL) queries are ubiquitous in modern software engineering. These queries can be costly when run on large databases with many entries and tables to consider. We propose using Genetic Improvement (GI) to explore patches for these queries, with the aim of optimising their execution time, whilst maintaining the functionality of the program in which they are utilised. Specifically, we propose three ways in which SQL JOIN statements can be mutated in order to improve performance. We also discuss the requirements of software being improved in this manner and the potential challenges of our approach.

show abstract