Federated Stream Processing Support for Real-Time Business Intelligence Applications

Botan, Irina; Cho, Younggoo; Derakhshan, Roozbeh; Dindar, Nihal; Haas, Laura M.; Kim, Kihong; Tatbul, Nesime

doi:10.1007/978-3-642-14559-9_2

Cited by 16 publications

(8 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Each site consists of a data centre with physical nodes running a local distributed stream processing system, and we assume seamless integration across all these systems at the federated sites [12]. Below, we provide a high-level overview of data stream processing in an FSPS.…”

Section: Federated Stream Processingmentioning

confidence: 99%

“…Tatbul [32] argues for the integration of multiple stream processing engines for a variety of applications and pinpoints the challenges when dealing with heterogeneous query semantics. Botan et al [12] present MaxStream, an FSPS for business intelligence applications. Our focus instead is on fairness in an overloaded FSPS using load shedding.…”

Section: Related Workmentioning

confidence: 99%

“…q and q (lines [12][13][14]. Given the lowest result SIC query q (line 12), it selects as many tuples as required from q so that q SIC reaches the result SIC value of the second lowest query q (lines [15][16].…”

Section: Single Node Balance-sic Fairnessmentioning

confidence: 99%

See 2 more Smart Citations

Themis

Kalyvianaki

Fiscato

Salonidis³

et al. 2016

Proceedings of the 2016 International Conference on Management of Data

View full text Add to dashboard Cite

Federated stream processing systems, which utilise nodes from multiple independent domains, can be found increasingly in multi-provider cloud deployments, internet-of-things systems, collaborative sensing applications and large-scale grid systems. To pool resources from several sites and take advantage of local processing, submitted queries are split into query fragments, which are executed collaboratively by different sites. When supporting many concurrent users, however, queries may exhaust available processing resources, thus requiring constant load shedding. Given that individual sites have autonomy over how they allocate query fragments on their nodes, it is an open challenge how to ensure global fairness on processing quality experienced by queries in a federated scenario.We describe THEMIS, a federated stream processing system for resource-starved, multi-site deployments. It executes queries in a globally fair fashion and provides users with constant feedback on the experienced processing quality for their queries. THEMIS associates stream data with its source information content (SIC), a metric that quantifies the contribution of that data towards the query result, based on the amount of source data used to generate it. We provide the BALANCE-SIC distributed load shedding algorithm that balances the SIC values of result data. Our evaluation shows that the BALANCE-SIC algorithm yields balanced SIC values across queries, as measured by Jain's Fairness Index. Our approach also incurs a low execution time overhead.

show abstract

Section: Federated Stream Processingmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Themis

Kalyvianaki

Fiscato

Salonidis³

et al. 2016

Proceedings of the 2016 International Conference on Management of Data

View full text Add to dashboard Cite

show abstract

“…Data stream technology (Botan, 2009) provides a good solution to build real-time data warehouse, with which increased refresh cycles to frequently update the data. This kind of data warehouse systems can achieve nearly real-time data updating, where the data latency typically is in the range from seconds to minutes.…”

Section: B Real-time Data Process Layermentioning

confidence: 99%

Enabling Real-Time Business Intelligence by Stream Data Mining

Fong¹,

Yang²

2011

New Fundamental Technologies in Data Mining

View full text Add to dashboard Cite

“…Modern digital source has a huge volume and consist of rapidly changing data. Data stream technology [4] provides a good solution to build real-time data warehouse, with which increased refresh cycles to frequently update the data. This kind of data warehouse systems can achieve near real-time data updating, where the data latency typically is in the range of seconds.…”

Section: B Real-time Data Process Layermentioning

confidence: 99%