AWESOME: Empowering Scalable Data Science on Social Media Data with an Optimized Tri-Store Data System

Abstract

Modern big data applications usually involve heterogeneous data sources and analytical functions, leading to increasing demand for polystore systems, especially analytical polystore systems. This paper presents AWESOME system along with a domain-specific language ADIL. ADIL is a powerful language which supports 1) native heterogeneous data models such as Corpus, Graph, and Relation; 2) a rich set of analytical functions; and 3) clear and rigorous semantics. AWESOME is an efficient tri-store middle-ware which 1) is built on the top of three heterogeneous DBMSs (Postgres, Solr, and Neo4j) and is easy to be extended to incorporate other systems; 2) supports the in-memory query engines and is equipped with analytical capability; 3) applies a cost model to efficiently execute workloads written in ADIL; 4) fully exploits machine resources to improve scalability. A set of experiments on real workloads demonstrate the capability, efficiency, and scalability of AWESOME.

Processing Analytical Queries over Polystore System for a Large Astronomy Data Repository

“…Polystore systems, particularly analytical Polystore systems, are becoming increasingly popular in modern big data applications because of their ability to handle a wide range of data sources and analytical tasks. ADIL, a dataflow language, is presented in this work [29], together with an AWESOME Polystore system. Powerful language ADIL supports heterogeneous data models such as Corpus and Graph; an extensive collection of analytical functions, and clear and strict semantics.…”