Fast OLAP query execution in main memory on large data in a cluster

Weidner, Martin; Dees, Jonathan; Sanders, Peter

doi:10.1109/bigdata.2013.6691616

Cited by 15 publications

(6 citation statements)

References 30 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Existing Works [10][11] Privacy and security of Big Data has become a primary concern as every piece of technology is generating huge volumes of data to process. Researchers have worked to find a solution for this problem and few solutions to the problem have been found.…”

Section: Security Mechanismsmentioning

confidence: 99%

Hadoop eco system for big data security and privacy

Adluru

Datla

Zhang

2015

2015 Long Island Systems, Applications and Technology

View full text Add to dashboard Cite

show abstract

Section: Security Mechanismsmentioning

confidence: 99%

Hadoop eco system for big data security and privacy

Adluru

Datla

Zhang

2015

2015 Long Island Systems, Applications and Technology

View full text Add to dashboard Cite

show abstract

“…The Query Engine: At the core of SAP HANA SOE is a high performance SQL-to-C code generation based query engine (HSQE) [9,22]. HSQE was initially released as a stand alone query engine embedded in SAP Lumira, a market leading data visualization software solution, and has since been extended for scale-out [29]. SAP HANA SOE further provides an optimizer that decomposes SQL queries into execution graphs suitable for running on a cluster of HSQE nodes.…”

Section: Components Of Sap Hana Soementioning

confidence: 99%

“…The HSQE cluster implements a DQP service managing the mapping from horizontal table partitions ("slices") to compute nodes. For an incoming query, the DQP service generates and globally optimizes a distributed execution plan that is specifically tailored for execution across the compute nodes in combination with efficient communication algorithms [30]. As outlined in Section 3.4, compute nodes are furthermore responsible to monitor the tail of the log and to apply ordered and timestamped updates to construct new versions, based on which queries are executed.…”

Section: Hana Soe Query Enginementioning

confidence: 99%

Towards scalable real-time analytics

et al. 2015

View full text Add to dashboard Cite

We present an overview of our work on the SAP HANA Scale-out Extension, a novel distributed database architecture designed to support large scale analytics over realtime data. This platform permits high performance OLAP with massive scale-out capabilities, while concurrently allowing OLTP workloads. This dual capability enables analytics over real-time changing data and allows fine grained userspecified service level agreements (SLAs) on data freshness. We advocate the decoupling of core database components such as query processing, concurrency control, and persistence, a design choice made possible by advances in highthroughput low-latency networks and storage devices. We provide full ACID guarantees and build on a logical timestamp mechanism to provide MVCC-based snapshot isolation, while not requiring synchronous updates of replicas. Instead, we use asynchronous update propagation guaranteeing consistency with timestamp validation. We provide a view into the design and development of a large scale data management platform for real-time analytics, driven by the needs of modern enterprise customers.

show abstract

“…Additionally, we generate plans for a distributed landscape. These plans can lead to strong speedup results compared to single machine execution as shown in [13] if the plans are specifically tailored for a clustered execution in combination with efficient communication algorithms. A first version of SAP HANA SOE was delivered to customers at the end of 2014 as part of SAP's analytic solution (Lumira Desktop and Lumira Teamserver).…”

Section: A Goals and Architectural Principlesmentioning

confidence: 99%

Towards a web-scale data management ecosystem demonstrated by SAP HANA

Faerber

Dees

Weidner

et al. 2015

2015 IEEE 31st International Conference on Data Engineering

Self Cite

View full text Add to dashboard Cite

Over the years, data management has diversified and moved into multiple directions, mainly caused by a significant growth in the application space with different usage patterns, a massive change in the underlying hardware characteristics, and-last but not least-growing data volumes to be processed. A solution matching these constraints has to cope with a multidimensional problem space including techniques dealing with a large number of domain-specific data types, data and consistency models, deployment scenarios, and processing, storage, and communication infrastructures on a hardware level. Specialized database engines are available and are positioned in the market optimizing a particular dimension on the one hand while relaxing other aspects (e.g. web-scale deployment with relaxed consistency). Today it is common sense, that there is no single engine which can handle all the different dimensions equally well and therefore we have very good reasons to tackle this problem and optimize the dimensions with specialized approaches in a first step. However, we argue for a second step (reflecting in our opinion on the even harder problem) of a deep integration of individual engines into a single coherent and consistent data management ecosystem providing not only shared components but also a common understanding of the overall business semantics. More specifically, a data management ecosystem provides common "infrastructure" for software and data life cycle management, backup/recovery, replication and high availability, accounting and monitoring, and many other operational topics, where administrators and users expect a harmonized experience. More importantly from an application perspective however, customer experience teaches us to provide a consistent business view across all different components and the ability to seamlessly combine different capabilities. For example, within recent customer-based Internet of Things scenarios, a huge potential exists in combining graphprocessing functionality with temporal and geospatial information and keywords extracted from high-throughput twitter streams. Using SAP HANA as the running example, we want to demonstrate what moving a set of individual engines and infra-structural components towards a holistic but also flexible data management ecosystem could look like. Although there are some solutions for some problems already visible on the horizon, we encourage the database research community in general to focus more on the Big Picture providing a holistic/integrated approach to efficiently deal with different types of data, with different access methods, and different consistency requirements-research in this field would push the envelope far beyond the traditional notion of data management.

show abstract

Fast OLAP query execution in main memory on large data in a cluster

Cited by 15 publications

References 30 publications

Hadoop eco system for big data security and privacy

Hadoop eco system for big data security and privacy

Towards scalable real-time analytics

Towards a web-scale data management ecosystem demonstrated by SAP HANA

Contact Info

Product

Resources

About