Orca

Soliman, Mohamed A.; Antova, Lyublena; Raghavan, Venkatesh; El-Helw, Amr; Gu, Zhongxian; Shen, Entong; Caragea, George C.; Garcia-Alvarado, Carlos; Rahman, Foyzur; Petropoulos, Michalis; Waas, Florian; Narayanan, S.; Krikellas, Konstantinos; Baldwin, Rhonda

doi:10.1145/2588555.2595637

Cited by 52 publications

(5 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…SAP [5,19] for example reports that their core data services contain over 100 views that reference more than 100 tables, with the largest view referencing over 4000 tables. Similarly, we hear reports from Tableau [39] and VMware [37] about auto generated queries that are dozens of pages of SQL.…”

Section: Discussionmentioning

confidence: 99%

“…After the classical approach in System R [31], Goetz Graefe pioneered the implementation of optimizations on relational algebra with the EXODUS [10], Volcano [8], and Cascades [9] systems. Modern optimizers like Calcite [3] or Orca [37] still use the same concepts. These systems all rely on operator centric optimizations that transform the plan with predefined rules.…”

Section: Related Workmentioning

confidence: 99%

“…The challenge for the optimizer here is that the data flow through the query, and its analysis can be astonishingly complex. Additionally, automatically generated queries with complex business logic amplify this problem [5,18,19,37]. Query optimizers struggle to deal with such complex input, which is especially painful for small datasets where query optimization can be more expensive than query execution.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Asymptotically Better Query Optimization Using Indexed Algebra

Fent,

Moerkotte,

Neumann

2023

Proc. VLDB Endow.

View full text Add to dashboard Cite

Query optimization is essential for the efficient execution of queries. The necessary analysis, if we can and should apply optimizations and transform the query plan, is already challenging. Traditional techniques focus on the availability of columns at individual operators, which does not scale for analysis of data flow through the query. Tracking available columns per operator takes quadratic space, which can result in multi-second optimization time for deep algebra trees. Instead, we need to re-think the naïve algebra representation to efficiently support data flow analysis. In this paper, we introduce Indexed Algebra , a novel representation of relational algebra that makes common optimization tasks efficient. Indexed Algebra enables efficient reasoning with an auxiliary index structure based on link/cut trees that support dynamic updates and queries in O (log n ). This approach not only improves the asymptotic complexity, but also allows elegant and concise formulations for the data flow questions needed for query optimization. While large queries see theoretically unbounded improvements, Indexed Algebra also improves optimization time of the relatively harmless queries of TPC-H and TPC-DS by more than 1.8×.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Asymptotically Better Query Optimization Using Indexed Algebra

Fent,

Moerkotte,

Neumann

2023

Proc. VLDB Endow.

View full text Add to dashboard Cite

show abstract

“…This proposal can be seen as an elegant adaptation of [39], proposed in a parallel database system, to a cloud system. More generally, with respect to the issue of query optimization in cloud environments, the most recent and relevant proposals are described in [11,41,53,61].…”

Section: Discussionmentioning

confidence: 99%

Big Data Management in the Cloud: Evolution or Crossroad?

Hameurlain

Morvan

2016

Communications in Computer and Information Science

View full text Add to dashboard Cite

OATAO is an open access repository that collects the work of Toulouse researchers and makes it freely available over the web where possible. Abstract. In this paper, we try to provide a synthetic and comprehensive state of the art concerning big data management in cloud environments. In this perspective, data management based on parallel and cloud (e.g. MapReduce) systems are overviewed, and compared by relying on meeting software requirements (e.g. data independence, software reuse), high performance, scalability, elasticity, and data availability. With respect to proposed cloud systems, we discuss evolution of their data manipulation languages and we try to learn some lessons should be exploited to ensure the viability of the next generation of large-scale data management systems for big data applications.

show abstract

“…Hence, we consider Impala v1.3.1 as our main competitor. Since Impala's compiler supports only a subset of TPC-DS queries [21], the evaluation of Impala utilizes the Impala TPC-DS Kit from Cloudera for re-writing queries, available at https://github.com/cloudera/impala-tpcds-kit. In addition, we also compare OceanRT against Hive-on-Tez v0.12, and we have published the query re-writing toolkit for Hive at https://github.com/simonzhangsm/hive-testbench.…”

Section: Methodsmentioning

confidence: 99%

Design and implementation of a real-time interactive analytics system for large spatio-temporal data

et al. 2014

View full text Add to dashboard Cite

In real-time interactive data analytics, the user expects to receive the results of each query within a short time period such as seconds. This is especially challenging when the data is big (e.g., on the scale of petabytes), and the analytics system runs on top of cloud infrastructure (e.g., thousands of interconnected commodity servers). We have been building such a system, called OceanRT, for managing large spatio-temporal data such as call logs and mobile web browsing records collected by a telecommunication company. Although there already exist systems for querying big data in real time, OceanRT's performance stands out due to several novel designs and components that address key efficiency and scalability issues that were largely overlooked in existing systems. First, OceanRT makes extensive use of software RDMA one-sided operations, which reduce networking costs without requiring specialized hardware. Second, OceanRT exploits the parallel computing capabilities of each node in the cloud through a novel architecture consisting of Access-Query Engines (AQEs) connected with minimal overhead. Third, OceanRT contains a novel storage scheme that optimizes for queries with joins and multi-dimensional selections, which are common for large spatiotemporal data. Experiments using the TPC-DS benchmark show that OceanRT is usually more than an order of magnitude faster than the current state-of-the-art systems.

show abstract

Orca

Cited by 52 publications

References 21 publications

Asymptotically Better Query Optimization Using Indexed Algebra

Asymptotically Better Query Optimization Using Indexed Algebra

Big Data Management in the Cloud: Evolution or Crossroad?

Design and implementation of a real-time interactive analytics system for large spatio-temporal data

Contact Info

Product

Resources

About