NebulaStream: Data Management for the Internet of Things

Zeuch, Steffen; Chatziliadis, Xenofon; Chaudhary, Ankit; Giouroukis, Dimitrios; Grulich, Philipp M.; Nugroho, Dwi Prasetyo Adi; Ziehn, Ariane; Mark, Volker

doi:10.1007/s13222-022-00415-0

Cited by 13 publications

(13 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The NebulaStream [73,74] platform offers an end-to-end data-management system for the IoT. It provides a unified environment for a sensor-fog-cloud infrastructure that handles heterogeneous hardware, unreliable nodes, and elastic network topology.…”

Section: Systems Integrationmentioning

confidence: 99%

“…In an experiment performed by Zeuch et al [73], the throughput of the Yahoo! Streaming Benchmark [70] was evaluated on on a RaspberryPi 3B+ using NebulaStream, Python, Flink, and a hand-optimized Java program.…”

Section: Systems Integrationmentioning

confidence: 99%

“…Future stream processing systems have to include multiple optimization techniques on operator, query, hardware, and application level to meet the high-throughput and lowlatency requirements of modern streaming applications. We are working towards this goal at BIFOLD by developing the NebulaStream System [73,74].…”

Section: Future Workmentioning

confidence: 99%

“…The approaches presented in Sect. 4 exploit advantages of modern hardware, such as multi-core processors and high- Cutty [14], Scotty [65,66,68], AJoin [34], Eco-Join [44], Zhang et al [76], Parallel ADWIN [26] Modern Hardware Zeuch et al [72], Grizzly [28], Slash [21] Distributed Window Processing AStream [35], Disco [8], Rhino [6,20] Stream Processing Systems Scotty Connectors to Apache Flink [13], Storm [61], Beam [3], Samza [47], Kafka Streams [38], Spark [60,71]; NebulaStream [73,74], Agora [69], SENSE [62,63,67] Applications…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Algorithms for Windowed Aggregations and Joins on Distributed Stream Processing Systems

et al. 2022

Self Cite

View full text Add to dashboard Cite

Window aggregations and windowed joins are central operators of modern real-time analytic workloads and significantly impact the performance of stream processing systems.This paper gives an overview of state-of-the-art research in this area conducted by the Berlin Institute for the Foundations of Learning and Data (BIFOLD) and the Technische Universität Berlin. To this end, we present different algorithms for efficiently processing windowed operators and discuss techniques for distributed stream processing. Recently, several approaches have leveraged modern hardware for windowed stream processing, which we will also include in this overview. Additionally, we describe the integration of windowed operators into various stream processing systems and diverse applications that use specialized window operations.

show abstract

Section: Systems Integrationmentioning

confidence: 99%

Section: Systems Integrationmentioning

confidence: 99%

Section: Future Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Algorithms for Windowed Aggregations and Joins on Distributed Stream Processing Systems

et al. 2022

Self Cite

View full text Add to dashboard Cite

show abstract

“…Their primary purpose is to transmit the collected data to monitoring stations for analysis. The processing performed on these devices is then streamed queries [38], [39], the majority of which is executed at the end of the chain in a centralized manner. Most use-cases consider sensors for the environment or for animal tracking and do not raise privacy issues.…”

Section: Related Workmentioning

confidence: 99%

Edgelet Computing: Pushing Query Processing and Liability at the Extreme Edge of the Network

Javet

Anciaux

Bouganim

et al. 2022

2022 22nd IEEE International Symposium on Cluster, Cloud and Internet Computing (CCGrid)

View full text Add to dashboard Cite

We call edgelet computing the current convergence between Opportunistic Network (OppNet) and Trusted Execution Environment (TEE) at the very edge of the network. We believe that this convergence bears the seeds of a novel and important class of applications leveraging fully decentralized and highly secure computations among data scattered on multiple personal devices. This paper introduces the Edgelet computing paradigm, defines properties that guarantee the safety, liveness and security of executions in this unusual context and proposes alternative strategies satisfying these properties. Preliminary performance evaluations and an ongoing real-case study highlights the practicality of the approach. Finally, the paper draws future research challenges for the database and distributed system community.

show abstract

Evolving the Digital Industrial Infrastructure for Production: Steps Taken and the Road Ahead

Pennekamp,

Belova,

Bergs

et al. 2023

Interdisciplinary Excellence Accelerator Series

View full text Add to dashboard Cite

The Internet of Production (IoP) leverages concepts such as digital shadows, data lakes, and a World Wide Lab (WWL) to advance today’s production. Consequently, it requires a technical infrastructure that can support the agile deployment of these concepts and corresponding high-level applications, which, e.g., demand the processing of massive data in motion and at rest. As such, key research aspects are the support for low-latency control loops, concepts on scalable data stream processing, deployable information security, and semantically rich and efficient long-term storage. In particular, such an infrastructure cannot continue to be limited to machines and sensors, but additionally needs to encompass networked environments: production cells, edge computing, and location-independent cloud infrastructures. Finally, in light of the envisioned WWL, i.e., the interconnection of production sites, the technical infrastructure must be advanced to support secure and privacy-preserving industrial collaboration. To evolve today’s production sites and lay the infrastructural foundation for the IoP, we identify five broad streams of research: (1) adapting data and stream processing to heterogeneous data from distributed sources, (2) ensuring data interoperability between systems and production sites, (3) exchanging and sharing data with different stakeholders, (4) network security approaches addressing the risks of increasing interconnectivity, and (5) security architectures to enable secure and privacy-preserving industrial collaboration. With our research, we evolve the underlying infrastructure from isolated, sparsely networked production sites toward an architecture that supports high-level applications and sophisticated digital shadows while facilitating the transition toward a WWL.

show abstract

NebulaStream: Data Management for the Internet of Things

Cited by 13 publications

References 17 publications

Algorithms for Windowed Aggregations and Joins on Distributed Stream Processing Systems

Algorithms for Windowed Aggregations and Joins on Distributed Stream Processing Systems

Edgelet Computing: Pushing Query Processing and Liability at the Extreme Edge of the Network

Evolving the Digital Industrial Infrastructure for Production: Steps Taken and the Road Ahead

Contact Info

Product

Resources

About